Nov 04 02:03:05.890574 ip-10-0-138-45 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 02:03:05.890586 ip-10-0-138-45 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 02:03:05.890592 ip-10-0-138-45 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 02:03:05.890893 ip-10-0-138-45 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 02:03:17.304546 ip-10-0-138-45 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 02:03:17.304562 ip-10-0-138-45 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 034ac16a29db4309b3300584587c2eae -- Nov 04 02:05:50.349596 ip-10-0-138-45 systemd[1]: Starting Kubernetes Kubelet... Nov 04 02:05:50.763735 ip-10-0-138-45 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:05:50.763735 ip-10-0-138-45 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 02:05:50.763735 ip-10-0-138-45 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:05:50.763735 ip-10-0-138-45 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:05:50.768531 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768450 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.768531 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768530 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.768531 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768534 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768537 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768541 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768544 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768546 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768549 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768551 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768556 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768560 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768563 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768566 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768568 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768571 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768573 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768576 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768579 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768581 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768584 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768586 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.768620 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768589 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768597 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768600 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768603 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768605 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768608 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768610 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768613 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768615 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768618 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768620 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768623 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768626 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768628 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768631 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768634 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768636 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768638 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768641 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768643 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.769066 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768646 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768648 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768650 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768653 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768655 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768658 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768660 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768662 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768665 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768667 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768669 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768672 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768675 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768677 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768682 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768685 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768687 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768690 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768693 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768695 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.769573 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768698 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768700 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768703 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768705 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768708 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768710 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768713 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768715 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768717 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768720 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768722 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768725 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768727 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768730 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768733 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768736 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768738 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768741 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768743 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768746 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.770075 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768748 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768751 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768753 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768759 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768762 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768765 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768768 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768771 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768773 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768776 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768779 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768781 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768784 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768786 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768789 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768791 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768794 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768796 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768799 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.770556 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768801 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768804 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768806 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768809 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768814 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.768818 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769598 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769605 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769610 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769613 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769615 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769618 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769621 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769624 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769626 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769629 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769632 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769635 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769638 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.771007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769641 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769644 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769648 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769651 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769653 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769656 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769658 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769661 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769663 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769666 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769668 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769671 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769673 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769676 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769678 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769681 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769683 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769686 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769688 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769691 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.771472 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769693 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769696 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769698 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769701 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769703 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769706 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769708 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769711 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769713 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769716 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769720 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769723 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769725 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769728 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769730 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769733 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769736 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769738 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769741 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769743 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.771945 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769746 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769748 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769751 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769753 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769756 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769758 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769761 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769764 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769767 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769769 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769772 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769775 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769778 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769780 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769783 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769785 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769788 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769791 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769793 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.772430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769796 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769798 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769801 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769804 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769806 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769809 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769811 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769814 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769817 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769820 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769822 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769825 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769827 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769829 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769832 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769834 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769837 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769839 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769842 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.772912 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769845 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769847 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769850 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769852 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769855 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769858 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769860 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769863 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769866 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769868 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769871 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769873 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769878 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769882 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.769885 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770357 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770362 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770365 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770368 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770370 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.773397 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770373 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770376 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770379 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770382 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770384 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770387 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770389 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770392 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770394 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770397 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770399 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770402 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770405 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770407 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770410 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770412 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770415 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770417 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770422 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.773969 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770424 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770427 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770430 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770433 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770435 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770438 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770440 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770443 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770446 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770448 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770451 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770453 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770456 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770459 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770461 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770463 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770466 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770469 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770471 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770474 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.774430 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770477 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770479 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770482 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770484 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770488 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770491 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770495 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770497 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770500 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770503 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770506 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770509 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770512 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770515 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770518 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770521 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770523 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770527 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770530 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.774897 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770532 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770535 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770537 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770540 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770542 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770545 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770547 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770550 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770552 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770555 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770559 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770563 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770566 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770568 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770571 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770573 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770576 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770578 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770581 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.775360 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770583 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770586 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770588 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770591 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770593 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770596 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770599 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770602 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770604 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770607 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770609 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770612 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770614 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770616 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770619 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770621 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770624 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770626 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770629 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.775810 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770631 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770633 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770636 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770638 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.770642 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771724 2570 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771732 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771746 2570 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771756 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771760 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771763 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771767 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771771 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771775 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771777 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771781 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771784 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771787 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771791 2570 flags.go:64] FLAG: --cgroup-root="" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771795 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771798 2570 flags.go:64] FLAG: --client-ca-file="" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771801 2570 flags.go:64] FLAG: --cloud-provider="external" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771804 2570 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771808 2570 flags.go:64] FLAG: --cluster-domain="" Nov 04 02:05:50.776279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771811 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771814 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771817 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771821 2570 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771824 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771827 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771830 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771833 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771836 2570 flags.go:64] FLAG: --contention-profiling="false" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771839 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771842 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771845 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771849 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771854 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771857 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771860 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771863 2570 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771866 2570 flags.go:64] FLAG: --enable-server="true" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771869 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771873 2570 flags.go:64] FLAG: --event-burst="100" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771876 2570 flags.go:64] FLAG: --event-qps="50" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771879 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771882 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771885 2570 flags.go:64] FLAG: --eviction-hard="" Nov 04 02:05:50.776860 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771890 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771892 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771896 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771900 2570 flags.go:64] FLAG: --eviction-soft="" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771903 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771906 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771909 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771912 2570 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771915 2570 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771917 2570 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771920 2570 flags.go:64] FLAG: --feature-gates="" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771924 2570 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771927 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771930 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771934 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771937 2570 flags.go:64] FLAG: --healthz-port="10248" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771940 2570 flags.go:64] FLAG: --help="false" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771943 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771946 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771949 2570 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771952 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771955 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771959 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 02:05:50.777481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771962 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771965 2570 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771968 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771971 2570 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771974 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771977 2570 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771979 2570 flags.go:64] FLAG: --kube-reserved="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771982 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771985 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771988 2570 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771991 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771994 2570 flags.go:64] FLAG: --lock-file="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.771997 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772000 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772003 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772008 2570 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772011 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772014 2570 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772016 2570 flags.go:64] FLAG: --logging-format="text" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772019 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772022 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772025 2570 flags.go:64] FLAG: --manifest-url="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772028 2570 flags.go:64] FLAG: --manifest-url-header="" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772032 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772035 2570 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 02:05:50.778035 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772038 2570 flags.go:64] FLAG: --max-pods="110" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772041 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772044 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772047 2570 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772050 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772053 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772056 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772059 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772065 2570 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772068 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772071 2570 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772074 2570 flags.go:64] FLAG: --pod-cidr="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772077 2570 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772079 2570 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772082 2570 flags.go:64] FLAG: --pods-per-core="0" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772085 2570 flags.go:64] FLAG: --port="10250" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772088 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772091 2570 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0f6a3e8edb0efabdd" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772094 2570 flags.go:64] FLAG: --qos-reserved="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772099 2570 flags.go:64] FLAG: --read-only-port="10255" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772102 2570 flags.go:64] FLAG: --register-node="true" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772105 2570 flags.go:64] FLAG: --register-with-taints="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772109 2570 flags.go:64] FLAG: --registry-burst="10" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772112 2570 flags.go:64] FLAG: --registry-qps="5" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772114 2570 flags.go:64] FLAG: --reserved-cpus="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772117 2570 flags.go:64] FLAG: --reserved-memory="" Nov 04 02:05:50.778687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772121 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772124 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772127 2570 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772129 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772132 2570 flags.go:64] FLAG: --runonce="false" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772135 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772138 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772140 2570 flags.go:64] FLAG: --seccomp-default="false" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772143 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772146 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772149 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772152 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772155 2570 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772157 2570 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772160 2570 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772163 2570 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772166 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772169 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772171 2570 flags.go:64] FLAG: --system-cgroups="" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772174 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772179 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772182 2570 flags.go:64] FLAG: --tls-cert-file="" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772185 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772188 2570 flags.go:64] FLAG: --tls-min-version="" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772191 2570 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 02:05:50.779497 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772195 2570 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772198 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772201 2570 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772203 2570 flags.go:64] FLAG: --v="2" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772207 2570 flags.go:64] FLAG: --version="false" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772211 2570 flags.go:64] FLAG: --vmodule="" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772215 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772218 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772389 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772394 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772397 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772400 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772402 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772405 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772408 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772410 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772413 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772415 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772418 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772420 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772423 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.780533 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772425 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772429 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772433 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772437 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772440 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772443 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772445 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772448 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772450 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772453 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772455 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772459 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772462 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772465 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772468 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772470 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772473 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772475 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772478 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772481 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.781067 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772483 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772486 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772488 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772491 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772493 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772496 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772498 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772501 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772503 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772505 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772508 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772510 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772513 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772515 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772518 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772521 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772523 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772526 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772528 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.781553 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772532 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772534 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772537 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772539 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772543 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772545 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772548 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772550 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772553 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772555 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772558 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772560 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772563 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772565 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772568 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772571 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772573 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772576 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772578 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.782007 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772580 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772584 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772588 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772591 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772594 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772597 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772599 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772602 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772604 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772606 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772609 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772612 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772614 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772618 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772620 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772623 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772625 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772629 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772631 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.782510 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772634 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772636 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772639 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772642 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772644 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772646 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772649 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772651 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772654 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772656 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772659 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772661 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772664 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772666 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772669 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772671 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.782954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.772696 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772731 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772736 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772739 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772742 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772744 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772747 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772750 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772755 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772758 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772760 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772764 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772766 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772769 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772773 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772775 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772778 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772780 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772782 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.783347 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772785 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772787 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772790 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772793 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772795 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772798 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772800 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772803 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772805 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772807 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772810 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772812 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772815 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772817 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772820 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772822 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772824 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772827 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772829 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772832 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.783798 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772834 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772837 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772840 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772842 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772846 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772848 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772851 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772854 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772857 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772859 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772863 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772882 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772886 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772888 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772891 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772893 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772896 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772899 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772901 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.784266 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772904 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772906 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772909 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772912 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772915 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772918 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772921 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772924 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772926 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772929 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772931 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772934 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772936 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772939 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772941 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772943 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772946 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772950 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772953 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772955 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.784739 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772959 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772961 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772964 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772966 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772969 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772971 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772974 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772976 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772978 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772981 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772983 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772986 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772988 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772991 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772993 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772996 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.772998 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773001 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773003 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.785253 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773005 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773008 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773011 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773014 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773016 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773018 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773021 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773023 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773026 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.773038 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.781654 2570 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.781671 2570 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.781691 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.781696 2570 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781756 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781760 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781763 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781766 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781768 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.785737 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781771 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781774 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781777 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781779 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781782 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781785 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781787 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781790 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781792 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781795 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781797 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781800 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781803 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781805 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781808 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781815 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781819 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781823 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781826 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781830 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.786307 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781833 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781836 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781839 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781842 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781845 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781848 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781851 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781854 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781857 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781859 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781862 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781864 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781867 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781869 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781872 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781874 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781877 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781879 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781882 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781885 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.786876 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781887 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781890 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781892 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781895 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781898 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781900 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781903 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781906 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781908 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781911 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781914 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781917 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781919 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781922 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781924 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781927 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781929 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781932 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781934 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.787382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781937 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781939 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781942 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781944 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781947 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781949 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781951 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781954 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781956 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781959 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781961 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781963 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781966 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781969 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781972 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781975 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781978 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781981 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781983 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.787854 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781986 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781988 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781990 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781993 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781996 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.781999 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782001 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782004 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782007 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782012 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782015 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782018 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782021 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782023 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782026 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782028 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782031 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782033 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782035 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782038 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.788316 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782040 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782043 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782046 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.782073 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782216 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782220 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782224 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782227 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782230 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782233 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782236 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782239 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782241 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782244 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782246 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782249 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.788793 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782252 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782254 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782256 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782259 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782262 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782264 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782267 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782269 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782272 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782274 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782277 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782279 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782282 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782284 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782286 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782289 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782291 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782294 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782296 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.789247 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782299 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782301 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782304 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782307 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782309 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782312 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782314 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782317 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782320 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782323 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782325 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782341 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782344 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782347 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782350 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782353 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782357 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782359 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782362 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.789812 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782365 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782367 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782370 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782372 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782375 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782377 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782380 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782382 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782385 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782387 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782390 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782392 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782395 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782398 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782403 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782406 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782409 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782411 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782414 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.790601 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782417 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782420 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782424 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782426 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782429 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782431 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782434 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782436 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782439 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782441 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782444 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782446 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782449 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782451 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782453 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782456 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782458 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782461 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782463 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782466 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.791172 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782468 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782471 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782474 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782476 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782479 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782481 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782483 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782486 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782488 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782491 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782494 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782497 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782499 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782502 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782505 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782513 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.791736 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782516 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.782540 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782573 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782577 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782580 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782583 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782586 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782588 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782591 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782593 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782596 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782598 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782601 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782603 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782606 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782608 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:05:50.792318 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782610 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782613 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782616 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782618 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782621 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782624 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782626 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782629 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782633 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782635 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782638 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782640 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782642 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782645 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782647 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782649 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782652 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782655 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782657 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:05:50.792725 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782659 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782662 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782664 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782667 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782669 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782672 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782674 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782677 2570 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782679 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782682 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782684 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782686 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782689 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782691 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782694 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782697 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782699 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782703 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782706 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:05:50.793175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782710 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782712 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782716 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782718 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782721 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782723 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782725 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782728 2570 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782730 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782732 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782735 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782738 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782741 2570 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782743 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782745 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782750 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782753 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782756 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782759 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:05:50.793896 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782761 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782763 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782766 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782768 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782771 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782773 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782775 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782778 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782780 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782783 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782785 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782787 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782790 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782792 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782795 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782798 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782800 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782803 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782806 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782808 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:05:50.794381 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782810 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782813 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782815 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782817 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782820 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782823 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782825 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782828 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782830 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782832 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782836 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782838 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782841 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782843 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:50.782845 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.782902 2570 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.784883 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.785838 2570 server.go:1014] "Starting client certificate rotation" Nov 04 02:05:50.794903 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.785949 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 02:05:50.795316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.786006 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 02:05:50.795316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.788864 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 02:05:50.795316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.790978 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 02:05:50.800231 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.800216 2570 log.go:25] "Validated CRI v1 runtime API" Nov 04 02:05:50.803392 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.803375 2570 log.go:25] "Validated CRI v1 image API" Nov 04 02:05:50.804263 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.804241 2570 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 02:05:50.806774 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.806755 2570 fs.go:135] Filesystem UUIDs: map[2bf98269-6ffb-4ebb-bfd8-22370da6d87e:/dev/nvme0n1p4 400bd14d-ef8a-4f68-8b76-24f7a16f5ad4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Nov 04 02:05:50.806842 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.806773 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 02:05:50.813574 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.813464 2570 manager.go:217] Machine: {Timestamp:2026-11-04 02:05:50.812232788 +0000 UTC m=+0.344005148 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100972 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec244f76d8a432ad5fc92bf111e8a982 SystemUUID:ec244f76-d8a4-32ad-5fc9-2bf111e8a982 BootID:034ac16a-29db-4309-b330-0584587c2eae Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:6a:27:f6:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:6a:27:f6:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:55:31:ba:e8:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 02:05:50.814398 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.814387 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 02:05:50.814521 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.814509 2570 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 02:05:50.815464 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815437 2570 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 02:05:50.815598 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815466 2570 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-45.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 02:05:50.815639 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815601 2570 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 02:05:50.815639 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815608 2570 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 02:05:50.815688 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815643 2570 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 02:05:50.815688 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.815659 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 02:05:50.816491 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.816481 2570 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 02:05:50.820372 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.820360 2570 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 02:05:50.820506 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.820496 2570 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 02:05:50.821373 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821362 2570 kubelet.go:493] "Attempting to sync node with API server" Nov 04 02:05:50.821405 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821377 2570 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 02:05:50.821405 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821388 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 02:05:50.821405 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821396 2570 kubelet.go:405] "Adding apiserver pod source" Nov 04 02:05:50.821405 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821404 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 02:05:50.821733 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.821722 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 02:05:50.824942 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.824924 2570 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 02:05:50.829376 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.829364 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 02:05:50.830247 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830236 2570 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 02:05:50.830289 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830266 2570 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 02:05:50.830437 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830427 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 02:05:50.830478 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830443 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 02:05:50.830478 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830449 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 02:05:50.830478 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830454 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 02:05:50.830478 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830459 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 02:05:50.830478 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830465 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 02:05:50.830923 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830913 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 02:05:50.830954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830924 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 02:05:50.830954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830933 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 02:05:50.830954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830940 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 02:05:50.830954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830948 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 02:05:50.831106 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.830956 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 02:05:50.831947 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.831937 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 02:05:50.831977 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.831950 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 02:05:50.835534 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.835518 2570 server.go:1263] "Started kubelet" Nov 04 02:05:50.835602 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.835572 2570 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 02:05:50.836126 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.836078 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 02:05:50.836223 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.836149 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 02:05:50.836378 ip-10-0-138-45 systemd[1]: Started Kubernetes Kubelet. Nov 04 02:05:50.837246 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.837169 2570 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 02:05:50.838004 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.837987 2570 server.go:317] "Adding debug handlers to kubelet server" Nov 04 02:05:50.840698 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.840682 2570 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 02:05:50.842235 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842218 2570 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 02:05:50.842362 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842223 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 02:05:50.842362 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.842324 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:05:50.842839 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842820 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 02:05:50.842920 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842873 2570 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 02:05:50.842920 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842886 2570 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 02:05:50.843038 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.842995 2570 reconstruct.go:97] "Volume reconstruction finished" Nov 04 02:05:50.843038 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843005 2570 reconciler.go:29] "Reconciler: start to sync state" Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843453 2570 factory.go:153] Registering CRI-O factory Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843473 2570 factory.go:223] Registration of the crio container factory successfully Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843525 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843532 2570 factory.go:55] Registering systemd factory Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843538 2570 factory.go:223] Registration of the systemd container factory successfully Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843549 2570 factory.go:103] Registering Raw factory Nov 04 02:05:50.843530 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.843558 2570 manager.go:1199] Started watching for new ooms in manager Nov 04 02:05:50.844556 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.844538 2570 manager.go:319] Starting recovery of all containers Nov 04 02:05:50.856591 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.856574 2570 manager.go:324] Recovery completed Nov 04 02:05:50.860323 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.860309 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:50.862109 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862092 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:50.862158 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862126 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:50.862158 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862141 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:50.862617 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862605 2570 cpu_manager.go:226] "Starting" policy="none" Nov 04 02:05:50.862617 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862615 2570 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 02:05:50.862703 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862630 2570 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 02:05:50.862703 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.862664 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:50.863581 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.863566 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:50.863648 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.863597 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:50.863648 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.863612 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:50.864833 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.864815 2570 policy_none.go:50] "Start" Nov 04 02:05:50.864833 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.864831 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 02:05:50.864976 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.864840 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 02:05:50.866887 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.866873 2570 policy_none.go:44] "Start" Nov 04 02:05:50.910019 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.909872 2570 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 02:05:50.913836 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.913824 2570 manager.go:341] "Starting Device Plugin manager" Nov 04 02:05:50.913924 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.913852 2570 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 02:05:50.913924 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.913864 2570 server.go:87] "Starting device plugin registration server" Nov 04 02:05:50.914095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.914083 2570 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 02:05:50.914147 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.914095 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 02:05:50.914198 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.914187 2570 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 02:05:50.914379 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.914268 2570 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 02:05:50.914379 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.914281 2570 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 02:05:50.914925 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.914907 2570 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 02:05:50.914987 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.914958 2570 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:05:50.969882 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.969852 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 02:05:50.972082 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.971369 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 02:05:50.972082 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.971389 2570 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 02:05:50.972082 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:50.971408 2570 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 02:05:50.972082 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:50.971451 2570 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 02:05:51.014798 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.014731 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:51.015707 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.015688 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:51.015788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.015720 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:51.015788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.015732 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:51.015788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.015762 2570 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.063519 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.063495 2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 02:05:51.063591 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.063576 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ttndr" Nov 04 02:05:51.072564 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.072543 2570 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal"] Nov 04 02:05:51.072619 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.072597 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:51.073469 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.073443 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:51.073520 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.073499 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:51.073556 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.073520 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:51.075709 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.075697 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:51.075915 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.075900 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.075988 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.075934 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:51.076546 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076529 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:51.076646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076557 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:51.076646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076562 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:51.076646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076580 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:51.076646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076587 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:51.076646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.076604 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:51.078788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.078774 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.078867 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.078802 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:05:51.079479 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.079462 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:05:51.079556 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.079491 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:05:51.079556 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.079530 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:05:51.101736 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:51.101719 2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.104354 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:51.104322 2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.145300 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.145274 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.145420 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.145305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.145420 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.145345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.150717 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.150701 2570 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:51.197315 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.197291 2570 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:51.197472 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.197324 2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:51.245629 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245601 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.245629 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245567 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.245786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.245786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.245786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245732 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.245786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.245743 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.249789 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.249770 2570 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:51.249865 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.249805 2570 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.249865 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:51.249831 2570 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-45.us-west-1.compute.internal\": node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:05:51.343052 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.343019 2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.403132 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.403102 2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:05:51.403954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.403938 2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.404190 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.404176 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.406996 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.406976 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:05:51.464127 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.464096 2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:05:51.785492 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.785470 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 02:05:51.785986 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.785593 2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 02:05:51.785986 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.785633 2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 02:05:51.785986 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.785635 2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 02:05:51.785986 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:51.785703 2570 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ac66bc7e1762247a7b68d1e1f0c05235-88944398bf9cb6fa.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-138-45.us-west-1.compute.internal.18e4b55a422b8461\": read tcp 10.0.138.45:34538->3.147.97.216:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-138-45.us-west-1.compute.internal.18e4b55a422b8461 default 5256 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-45.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-45.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-45.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:50 +0000 UTC,LastTimestamp:2026-11-04 02:05:51.015725848 +0000 UTC m=+0.547498209,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-45.us-west-1.compute.internal,}" Nov 04 02:05:51.822563 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.822540 2570 apiserver.go:52] "Watching apiserver" Nov 04 02:05:51.843215 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:51.843188 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 02:05:52.068309 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.068221 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:00:51 +0000 UTC" deadline="2028-04-28 21:06:25.097049969 +0000 UTC" Nov 04 02:05:52.068309 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.068252 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13003h0m33.028802459s" Nov 04 02:05:52.097137 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.097107 2570 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 02:05:52.097498 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.097478 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-shp2z","openshift-ovn-kubernetes/ovnkube-node-t2htq","openshift-cluster-node-tuning-operator/tuned-pfcpv","openshift-dns/node-resolver-b55hv","openshift-multus/multus-additional-cni-plugins-z7kbm","openshift-multus/multus-ghkfg","kube-system/global-pull-secret-syncer-tjxq6","kube-system/konnectivity-agent-g5dnk","kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal","openshift-image-registry/node-ca-2gh6q","openshift-multus/network-metrics-daemon-5h9d5","openshift-network-diagnostics/network-check-target-56kwr"] Nov 04 02:05:52.099916 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.099897 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.099990 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.099960 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:05:52.101816 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.101801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.101895 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.101846 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:05:52.103871 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.103852 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.106016 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.105997 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.108185 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.108168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.110359 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.110342 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.112541 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.112523 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:52.112623 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.112582 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:05:52.114597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.114583 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.116833 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.116815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.119045 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.119027 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.121258 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.121241 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.123548 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.123531 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.145174 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.144918 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149349 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149403 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxlk6\" (UniqueName: \"kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149428 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149476 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149491 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhvjf\" (UniqueName: \"kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149557 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149605 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149637 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.149854 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149658 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149712 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149782 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149834 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149854 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b24rv\" (UniqueName: \"kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149891 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnzrd\" (UniqueName: \"kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149905 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.150531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149978 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.149991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddzdg\" (UniqueName: \"kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150171 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150184 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150236 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnczh\" (UniqueName: \"kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxmm2\" (UniqueName: \"kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.151136 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150365 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150407 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150479 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150550 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150572 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150601 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150624 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150646 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmnhc\" (UniqueName: \"kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.151601 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150784 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150800 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150842 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljpv\" (UniqueName: \"kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150869 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150886 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150968 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.150992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.151014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.151037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.151060 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.151082 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.152061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.151106 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.165233 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.165212 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-db624\"" Nov 04 02:05:52.165314 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.165257 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ckptg\"" Nov 04 02:05:52.165314 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.165287 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kvkfc\"" Nov 04 02:05:52.165418 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.165353 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-szjlx\"" Nov 04 02:05:52.166042 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.166024 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rtznt\"" Nov 04 02:05:52.168579 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.168560 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 02:05:52.168752 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.168741 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qq5vw\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222032 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222060 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222091 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222099 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222109 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222125 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222124 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222178 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222180 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222374 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222382 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222392 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222392 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222415 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222423 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222447 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222457 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222465 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222476 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222486 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222511 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222542 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 02:05:52.222711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222702 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 02:05:52.223104 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.222787 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 02:05:52.249838 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.249820 2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 02:05:52.249914 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.249874 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dh9wt" Nov 04 02:05:52.251989 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.251974 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252040 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.251997 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252040 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252013 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252040 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxlk6\" (UniqueName: \"kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.252159 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252069 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252159 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.252159 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252094 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252159 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252094 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252159 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252209 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252233 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvjf\" (UniqueName: \"kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252222 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252259 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252301 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252316 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.252396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252307 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252381 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.252430 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252433 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.252524 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:52.752481161 +0000 UTC m=+2.284253528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252553 2570 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252579 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252603 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252642 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252655 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.252832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252701 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252688 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252811 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252860 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24rv\" (UniqueName: \"kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252862 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252873 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252833 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252890 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252904 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252923 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252926 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252966 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.252997 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnzrd\" (UniqueName: \"kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.253597 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253060 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253089 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253128 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253144 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253169 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253174 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253180 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253220 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253224 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253242 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253269 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253314 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253312 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253362 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.254322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253371 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253390 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzdg\" (UniqueName: \"kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnczh\" (UniqueName: \"kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253582 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253589 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253623 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxmm2\" (UniqueName: \"kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253635 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253648 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253651 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253743 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253796 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253803 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253797 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253825 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253833 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253850 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253876 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253931 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253952 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253962 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.253977 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254002 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254027 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.255724 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254035 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254038 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.254053 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.254123 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:05:52.754091555 +0000 UTC m=+2.285863906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254118 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254243 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254274 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmnhc\" (UniqueName: \"kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254345 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254397 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254446 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256296 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254457 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254468 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljpv\" (UniqueName: \"kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254413 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254529 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254548 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254711 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254784 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254835 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254859 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.254943 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.256834 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255097 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255163 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255206 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255235 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255255 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255270 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255298 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.255360 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.256432 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.256461 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.256562 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.257316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.256614 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.265251 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.265236 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 02:05:52.265522 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.265510 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kn9lr\"" Nov 04 02:05:52.265587 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.265527 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z899q\"" Nov 04 02:05:52.265636 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.265612 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 02:05:52.265636 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.265619 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h6pb6\"" Nov 04 02:05:52.277500 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.277078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.314152 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.314115 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxlk6\" (UniqueName: \"kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.314299 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.314210 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzdg\" (UniqueName: \"kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.314651 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.314628 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24rv\" (UniqueName: \"kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.314800 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.314766 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:05:52.314800 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.314786 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:05:52.314800 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.314800 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:52.315023 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.314857 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:52.814843299 +0000 UTC m=+2.346615660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:52.315023 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.314915 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvjf\" (UniqueName: \"kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.315174 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.315157 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnczh\" (UniqueName: \"kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.317728 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.317703 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljpv\" (UniqueName: \"kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.318284 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.318266 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxmm2\" (UniqueName: \"kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.318619 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.318575 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmnhc\" (UniqueName: \"kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.321800 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.321777 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnzrd\" (UniqueName: \"kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.435784 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.435752 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:05:52.450315 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.450292 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b55hv" Nov 04 02:05:52.476675 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.476650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:05:52.481666 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.481647 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ghkfg" Nov 04 02:05:52.494216 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.494185 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94607daadef9ea3ca6175efce35c3d4a.slice/crio-02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46 WatchSource:0}: Error finding container 02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46: Status 404 returned error can't find the container with id 02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46 Nov 04 02:05:52.494692 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.494666 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50e239d_68ef_4cde_88ab_ee15083433d2.slice/crio-c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80 WatchSource:0}: Error finding container c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80: Status 404 returned error can't find the container with id c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80 Nov 04 02:05:52.495175 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.495154 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ceb3d0c15f7c179f81071c0e8686a7.slice/crio-da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2 WatchSource:0}: Error finding container da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2: Status 404 returned error can't find the container with id da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2 Nov 04 02:05:52.495650 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.495630 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:05:52.495759 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.495739 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30429b8d_d02a_4d8c_a500_a3bbf17e7a61.slice/crio-49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9 WatchSource:0}: Error finding container 49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9: Status 404 returned error can't find the container with id 49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9 Nov 04 02:05:52.498899 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.498884 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:05:52.504805 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.504789 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb61a6a8_f7ee_4140_8dff_1b292b2173b8.slice/crio-a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621 WatchSource:0}: Error finding container a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621: Status 404 returned error can't find the container with id a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621 Nov 04 02:05:52.523901 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.523884 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:05:52.529649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.529635 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:05:52.533292 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.533268 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1f5f84_c79a_47fe_9b41_f6d8d7b10100.slice/crio-050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2 WatchSource:0}: Error finding container 050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2: Status 404 returned error can't find the container with id 050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2 Nov 04 02:05:52.537781 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.537759 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d28c97_8c3a_4ccf_8156_c496da17cbea.slice/crio-6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49 WatchSource:0}: Error finding container 6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49: Status 404 returned error can't find the container with id 6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49 Nov 04 02:05:52.539131 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.539117 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:52.544600 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.544584 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:05:52.547432 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.547413 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9507ca_eeeb_4ed3_ab31_019380293e75.slice/crio-180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14 WatchSource:0}: Error finding container 180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14: Status 404 returned error can't find the container with id 180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14 Nov 04 02:05:52.552859 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:52.552840 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode114adad_34e0_4007_9a4c_011c60df73c6.slice/crio-66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc WatchSource:0}: Error finding container 66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc: Status 404 returned error can't find the container with id 66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc Nov 04 02:05:52.757639 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.757597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:52.757812 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.757649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:52.757812 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.757777 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:52.757812 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.757783 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:52.757937 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.757844 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.75782902 +0000 UTC m=+3.289601399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:52.757937 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.757863 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.757854881 +0000 UTC m=+3.289627230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:52.827652 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.827619 2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:52.858954 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.858924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:52.859110 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.859091 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:05:52.859167 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.859118 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:05:52.859167 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.859132 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:52.859243 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:52.859194 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.859176724 +0000 UTC m=+3.390949088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:52.878927 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.878907 2570 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:52.976116 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.976025 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2gh6q" event={"ID":"e114adad-34e0-4007-9a4c-011c60df73c6","Type":"ContainerStarted","Data":"66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc"} Nov 04 02:05:52.977101 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.977079 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g5dnk" event={"ID":"fc9507ca-eeeb-4ed3-ab31-019380293e75","Type":"ContainerStarted","Data":"180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14"} Nov 04 02:05:52.978670 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.978654 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ghkfg" event={"ID":"30429b8d-d02a-4d8c-a500-a3bbf17e7a61","Type":"ContainerStarted","Data":"49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9"} Nov 04 02:05:52.979741 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.979721 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerStarted","Data":"c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80"} Nov 04 02:05:52.980712 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.980693 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"33ceb3d0c15f7c179f81071c0e8686a7","Type":"ContainerStarted","Data":"da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2"} Nov 04 02:05:52.981705 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.981677 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerStarted","Data":"02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46"} Nov 04 02:05:52.983861 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.983839 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" event={"ID":"72d28c97-8c3a-4ccf-8156-c496da17cbea","Type":"ContainerStarted","Data":"6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49"} Nov 04 02:05:52.985631 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.985608 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2"} Nov 04 02:05:52.986618 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:52.986597 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shp2z" event={"ID":"bb61a6a8-f7ee-4140-8dff-1b292b2173b8","Type":"ContainerStarted","Data":"a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621"} Nov 04 02:05:53.254320 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.254230 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:52 +0000 UTC" deadline="2028-05-25 11:57:36.4178496 +0000 UTC" Nov 04 02:05:53.254320 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.254263 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13641h51m43.163590625s" Nov 04 02:05:53.257324 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.257306 2570 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:05:53.315714 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:53.315681 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803bdfd3_1098_4104_abc0_91f2b5ec5192.slice/crio-269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75 WatchSource:0}: Error finding container 269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75: Status 404 returned error can't find the container with id 269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75 Nov 04 02:05:53.334388 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:05:53.334358 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66fdb210_e77b_4433_a0c3_0ac8262cdcb9.slice/crio-623ed76e5d8b6c886a3011a1ef3612788029017db6abbdae0cc27016d9da93dc WatchSource:0}: Error finding container 623ed76e5d8b6c886a3011a1ef3612788029017db6abbdae0cc27016d9da93dc: Status 404 returned error can't find the container with id 623ed76e5d8b6c886a3011a1ef3612788029017db6abbdae0cc27016d9da93dc Nov 04 02:05:53.766578 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.766538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:53.766786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.766618 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:53.766786 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.766758 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:53.766915 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.766813 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:05:55.766799356 +0000 UTC m=+5.298571719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:53.766915 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.766874 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:53.766915 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.766913 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:55.766900096 +0000 UTC m=+5.298672459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:53.867271 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.867047 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:53.867271 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.867246 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:05:53.867271 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.867277 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:05:53.868032 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.867292 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:53.868032 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.867379 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:55.867356375 +0000 UTC m=+5.399128741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.972375 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.972514 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.972601 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.972679 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:53.972742 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:53.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:53.972837 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:05:54.006607 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:54.006477 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" event={"ID":"66fdb210-e77b-4433-a0c3-0ac8262cdcb9","Type":"ContainerStarted","Data":"623ed76e5d8b6c886a3011a1ef3612788029017db6abbdae0cc27016d9da93dc"} Nov 04 02:05:54.016926 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:54.016794 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b55hv" event={"ID":"803bdfd3-1098-4104-abc0-91f2b5ec5192","Type":"ContainerStarted","Data":"269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75"} Nov 04 02:05:54.255545 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:54.255455 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:52 +0000 UTC" deadline="2028-06-16 17:34:44.606819081 +0000 UTC" Nov 04 02:05:54.255545 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:54.255516 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14175h28m50.351307775s" Nov 04 02:05:55.785474 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.785428 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:55.786004 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.785509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:55.786004 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.785604 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:55.786004 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.785633 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:55.786004 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.785688 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:59.785665001 +0000 UTC m=+9.317437354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:55.786004 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.785714 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:05:59.785702241 +0000 UTC m=+9.317474594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:55.886980 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.886943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:55.887173 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.887138 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:05:55.887173 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.887159 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:05:55.887173 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.887173 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:55.887344 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.887239 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:59.887218268 +0000 UTC m=+9.418990633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:55.973344 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.973308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:55.973536 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.973474 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:05:55.973615 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.973545 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:55.973664 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.973625 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:05:55.973727 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:55.973670 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:55.973778 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:55.973752 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:05:57.046426 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:57.046289 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerStarted","Data":"55b00b168ff0cd126f24415ccf0879291121721cb2f3a65f5f53a4bcf2992298"} Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:57.972201 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:57.972425 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:57.972517 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:57.972605 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:57.972663 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:57.972770 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:57.972727 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:05:58.055787 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.055745 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g5dnk" event={"ID":"fc9507ca-eeeb-4ed3-ab31-019380293e75","Type":"ContainerStarted","Data":"4430ee54f8562d7d70ec518327c4b5a3a05d1fa84285190afebc66b1a0e2bcc4"} Nov 04 02:05:58.059988 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.059194 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="55b00b168ff0cd126f24415ccf0879291121721cb2f3a65f5f53a4bcf2992298" exitCode=0 Nov 04 02:05:58.059988 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.059280 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"55b00b168ff0cd126f24415ccf0879291121721cb2f3a65f5f53a4bcf2992298"} Nov 04 02:05:58.065276 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.064828 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"33ceb3d0c15f7c179f81071c0e8686a7","Type":"ContainerStarted","Data":"13fb33366fb6ecf6849f03da50e2f95ba14dabe67902d7e7a422e6b9bed21032"} Nov 04 02:05:58.068504 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.068375 2570 generic.go:356] "Generic (PLEG): container finished" podID="94607daadef9ea3ca6175efce35c3d4a" containerID="ab790b2418747a40376f998ae8b2b08b7a4d51e83e3e51e4818de08f7391ce93" exitCode=0 Nov 04 02:05:58.068504 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.068417 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerDied","Data":"ab790b2418747a40376f998ae8b2b08b7a4d51e83e3e51e4818de08f7391ce93"} Nov 04 02:05:58.181738 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.181661 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g5dnk" podStartSLOduration=2.46310596 podStartE2EDuration="7.181645253s" totalImagesPullingTime="4.718539293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:58.18154553 +0000 UTC m=+7.713317903" watchObservedRunningTime="2026-11-04 02:05:58.181645253 +0000 UTC m=+7.713417633" Nov 04 02:05:58.560320 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.559523 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" podStartSLOduration=6.559503159 podStartE2EDuration="6.559503159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:58.559247408 +0000 UTC m=+8.091019781" watchObservedRunningTime="2026-11-04 02:05:58.559503159 +0000 UTC m=+8.091275534" Nov 04 02:05:58.814764 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.814545 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:58.815360 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:58.815158 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:05:59.077010 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.076929 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerStarted","Data":"31b3ed88aff40fa8ece592ac216c031d62dfb33c713b635f28b35f8c9292cd3c"} Nov 04 02:05:59.830787 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.830746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:59.830970 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.830833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:59.831034 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.830974 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:59.831092 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.831041 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.83102103 +0000 UTC m=+17.362793396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:05:59.831153 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.831120 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:59.831205 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.831166 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.831153268 +0000 UTC m=+17.362925635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:05:59.931885 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.931847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:59.932049 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.932001 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:05:59.932049 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.932019 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:05:59.932049 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.932028 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:59.932222 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.932075 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.932059905 +0000 UTC m=+17.463832253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:05:59.972647 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.972565 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:05:59.972647 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.972609 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:05:59.972881 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:05:59.972566 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:05:59.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.972727 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:05:59.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.972782 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:05:59.972881 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:05:59.972849 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:00.081353 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:00.081305 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2gh6q" event={"ID":"e114adad-34e0-4007-9a4c-011c60df73c6","Type":"ContainerStarted","Data":"cf4503dd7a82ed5a214878ef8095ed6a65cbacfd58f23bdf03f3a05f9e486aba"} Nov 04 02:06:00.086972 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:00.086940 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" event={"ID":"72d28c97-8c3a-4ccf-8156-c496da17cbea","Type":"ContainerStarted","Data":"10fbfbd0e7cb1be51d9c638df3f4f349d8ce744ccec4c78c56270fccd505b20b"} Nov 04 02:06:00.087118 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:00.087037 2570 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 02:06:00.205563 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:00.205496 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" podStartSLOduration=8.205471625 podStartE2EDuration="8.205471625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:59.218474028 +0000 UTC m=+8.750246402" watchObservedRunningTime="2026-11-04 02:06:00.205471625 +0000 UTC m=+9.737244001" Nov 04 02:06:00.205742 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:00.205611 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2gh6q" podStartSLOduration=2.00921101 podStartE2EDuration="9.205604136s" totalImagesPullingTime="7.196393126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:00.204258091 +0000 UTC m=+9.736030467" watchObservedRunningTime="2026-11-04 02:06:00.205604136 +0000 UTC m=+9.737376510" Nov 04 02:06:01.972225 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:01.972186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:01.972715 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:01.972346 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:01.972715 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:01.972421 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:01.972715 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:01.972492 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:01.972715 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:01.972538 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:01.972715 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:01.972611 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:02.096455 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.096008 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ghkfg" event={"ID":"30429b8d-d02a-4d8c-a500-a3bbf17e7a61","Type":"ContainerStarted","Data":"d77b063fa3dedb9f6730a8a861cfbacf3dcfa4a2e8116506f4e97cc75a2ee18c"} Nov 04 02:06:02.224856 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.224752 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-ghkfg" podStartSLOduration=1.9146550979999999 podStartE2EDuration="11.224734048s" totalImagesPullingTime="9.31007895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:02.223893358 +0000 UTC m=+11.755665726" watchObservedRunningTime="2026-11-04 02:06:02.224734048 +0000 UTC m=+11.756506418" Nov 04 02:06:02.387156 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.387112 2570 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 02:06:02.920522 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.920357 2570 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:06:02.387134986Z","UUID":"1d389176-0895-44c7-a157-928f90286913","Handler":null,"Name":"","Endpoint":""} Nov 04 02:06:02.922906 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.922876 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 02:06:02.923059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:02.922929 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 02:06:03.100783 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.100750 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" event={"ID":"72d28c97-8c3a-4ccf-8156-c496da17cbea","Type":"ContainerStarted","Data":"c7a05f29599301e1a0cd62e2ab05744769ed6da98989f839186441e9ab91c809"} Nov 04 02:06:03.102525 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.102490 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b55hv" event={"ID":"803bdfd3-1098-4104-abc0-91f2b5ec5192","Type":"ContainerStarted","Data":"7307163b3e71ea3090c4e87db798eb0d8258a5527e4389b924452e821f1cb36f"} Nov 04 02:06:03.105126 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.105099 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="4c01fa9cd28643c6b38025caf4db4964d123dd35b29467b607132710c81b8cb5" exitCode=0 Nov 04 02:06:03.106066 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.106033 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"4c01fa9cd28643c6b38025caf4db4964d123dd35b29467b607132710c81b8cb5"} Nov 04 02:06:03.217676 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.217366 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-b55hv" podStartSLOduration=3.446511322 podStartE2EDuration="12.21732593s" totalImagesPullingTime="8.770814608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:03.216825772 +0000 UTC m=+12.748598140" watchObservedRunningTime="2026-11-04 02:06:03.21732593 +0000 UTC m=+12.749098300" Nov 04 02:06:03.971850 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.971823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:03.972046 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.971868 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:03.972046 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:03.971942 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:03.972046 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:03.971972 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:03.972046 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:03.972023 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:03.972212 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:03.972115 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:04.109550 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:04.109513 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shp2z" event={"ID":"bb61a6a8-f7ee-4140-8dff-1b292b2173b8","Type":"ContainerStarted","Data":"7d09ea56fadddc990189f1e5e5b4244f935b55088343eb6aa3eee41df61e6ffc"} Nov 04 02:06:04.222794 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:04.222698 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-shp2z" podStartSLOduration=3.815679933 podStartE2EDuration="13.222684155s" totalImagesPullingTime="9.407004222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:04.22266724 +0000 UTC m=+13.754439610" watchObservedRunningTime="2026-11-04 02:06:04.222684155 +0000 UTC m=+13.754456516" Nov 04 02:06:04.755730 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:04.755635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g5dnk_fc9507ca-eeeb-4ed3-ab31-019380293e75/konnectivity-agent/0.log" limit=1024 Nov 04 02:06:05.113862 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.113827 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" event={"ID":"72d28c97-8c3a-4ccf-8156-c496da17cbea","Type":"ContainerStarted","Data":"fa0d9fee04dce045e48601537dc2747ca6c35cac27508f89c9077a132b99c36e"} Nov 04 02:06:05.115537 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.115507 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" event={"ID":"66fdb210-e77b-4433-a0c3-0ac8262cdcb9","Type":"ContainerStarted","Data":"5dc9828308a5b9aa5ac1220b9842e47e96d7cdce49a20b5aa17dd0143a317634"} Nov 04 02:06:05.117781 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.117753 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="e053aec983bc8a7e29ee760bba38cbf7520627369b6d7816c08392b77ee5d934" exitCode=0 Nov 04 02:06:05.117883 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.117835 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"e053aec983bc8a7e29ee760bba38cbf7520627369b6d7816c08392b77ee5d934"} Nov 04 02:06:05.347659 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.347605 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" podStartSLOduration=2.964225528 podStartE2EDuration="14.347587366s" totalImagesPullingTime="11.383361838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:05.34735012 +0000 UTC m=+14.879122486" watchObservedRunningTime="2026-11-04 02:06:05.347587366 +0000 UTC m=+14.879359737" Nov 04 02:06:05.347998 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.347973 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" podStartSLOduration=2.978659446 podStartE2EDuration="14.347967924s" totalImagesPullingTime="11.369308478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:05.230629115 +0000 UTC m=+14.762401485" watchObservedRunningTime="2026-11-04 02:06:05.347967924 +0000 UTC m=+14.879740323" Nov 04 02:06:05.971912 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.971867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:05.972063 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.971915 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:05.972113 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:05.972076 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:05.972113 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:05.972089 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:05.972176 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:05.972159 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:05.972230 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:05.972205 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:06.913386 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:06.913311 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b55hv_803bdfd3-1098-4104-abc0-91f2b5ec5192/dns-node-resolver/0.log" Nov 04 02:06:07.125056 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.125021 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="9a21ac436fc36befa309df209751a3d1b4c47968fc997424893211d8c266d67b" exitCode=0 Nov 04 02:06:07.125222 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.125079 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"9a21ac436fc36befa309df209751a3d1b4c47968fc997424893211d8c266d67b"} Nov 04 02:06:07.711895 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.711683 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2gh6q_e114adad-34e0-4007-9a4c-011c60df73c6/node-ca/0.log" Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.895319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.895394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.895489 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.895493 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.895544 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:06:23.895532222 +0000 UTC m=+33.427304568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:06:07.895798 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.895594 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:23.895566986 +0000 UTC m=+33.427339349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:06:07.972006 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.971910 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:07.972006 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.971962 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:07.972541 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.971911 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:07.972541 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.972032 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:07.972541 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.972170 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:07.972541 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.972262 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:07.996180 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:07.996151 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:07.996325 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.996290 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:06:07.996325 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.996306 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:06:07.996325 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.996315 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:06:07.996468 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:07.996389 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:23.996377113 +0000 UTC m=+33.528149459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:06:09.132532 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.132505 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"1bfc4842262a9ee812dea3a34e5b0b7ffcf48c930c5ab2cefeacd7ac13f0d9e3"} Nov 04 02:06:09.132849 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.132576 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"c0e1828c78bf5e617417b298bcafbf59b78412e7b13d9e4ef652a13567039b84"} Nov 04 02:06:09.132849 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.132611 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"6f56f185588ab64c65aa40f8bf828c5dbf622d2e2f5cbd6e917859a59d87bfd5"} Nov 04 02:06:09.972059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.971889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:09.972243 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.971913 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:09.972243 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:09.972123 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:09.972243 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:09.971954 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:09.972243 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:09.972234 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:09.972430 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:09.972267 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:10.137003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:10.136972 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"6f615549a2a66b20abeb3c62022ae86e86d7368bd5aa0f8f239c093e4002d86d"} Nov 04 02:06:10.137461 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:10.137042 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"a54517048b66934d6db85ac76d54f3807e42c29b358dd043997312fd598a6994"} Nov 04 02:06:10.137461 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:10.137083 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"c611165bf53c3e429aebb87b4832d9bff603aba1c13bc5788b89a1e1dfecdd07"} Nov 04 02:06:10.139524 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:10.139502 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="001619a6c6e1523db6e43396b3d146b06113af48c58dc98329f707b45d2d3348" exitCode=0 Nov 04 02:06:10.139611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:10.139535 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"001619a6c6e1523db6e43396b3d146b06113af48c58dc98329f707b45d2d3348"} Nov 04 02:06:11.144768 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:11.144733 2570 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="e859ea26f2a692df63cc0a19b4c80b529c325bec1807bbf0c832b3d48cad90c4" exitCode=0 Nov 04 02:06:11.145221 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:11.144790 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerDied","Data":"e859ea26f2a692df63cc0a19b4c80b529c325bec1807bbf0c832b3d48cad90c4"} Nov 04 02:06:11.972735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:11.972690 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:11.972735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:11.972724 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:11.972983 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:11.972732 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:11.972983 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:11.972798 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:11.972983 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:11.972867 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:11.972983 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:11.972940 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:12.149937 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:12.149904 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"87b6d0e1fd9c13d9c17ed63f9b2101ccea95de361ca8989c4331f5b66857ac35"} Nov 04 02:06:12.153187 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:12.153160 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" event={"ID":"a50e239d-68ef-4cde-88ab-ee15083433d2","Type":"ContainerStarted","Data":"9ea6637d566b3445e1a9da68f2e1a91eb94c0d2a2350912712bcf62b4f8de65b"} Nov 04 02:06:12.273817 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:12.273770 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z7kbm" podStartSLOduration=7.701447776 podStartE2EDuration="21.273754775s" totalImagesPullingTime="13.572306999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:12.273455706 +0000 UTC m=+21.805228075" watchObservedRunningTime="2026-11-04 02:06:12.273754775 +0000 UTC m=+21.805527144" Nov 04 02:06:13.971584 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:13.971554 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:13.972191 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:13.971651 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:13.972191 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:13.971664 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:13.972191 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:13.971680 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:13.972191 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:13.971745 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:13.972191 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:13.971806 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:14.160061 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:14.159847 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" event={"ID":"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100","Type":"ContainerStarted","Data":"974bf810f2e28900c896e09b9239658b5e9df38444b585aeb58915d83b536550"} Nov 04 02:06:14.160221 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:14.160112 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:14.160221 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:14.160165 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:14.176322 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:14.176300 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:14.434714 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:14.434668 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" podStartSLOduration=7.041612333 podStartE2EDuration="23.434655971s" totalImagesPullingTime="16.393043638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:14.385555187 +0000 UTC m=+23.917327555" watchObservedRunningTime="2026-11-04 02:06:14.434655971 +0000 UTC m=+23.966428339" Nov 04 02:06:15.162156 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:15.162120 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:15.177587 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:15.177564 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:15.971998 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:15.971969 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:15.971998 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:15.971987 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:15.972201 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:15.972103 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:15.972201 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:15.972139 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:15.972263 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:15.972221 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:15.972295 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:15.972277 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:17.972187 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:17.972142 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:17.972187 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:17.972167 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:17.972729 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:17.972265 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:17.972729 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:17.972295 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:17.972729 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:17.972388 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:17.972729 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:17.972491 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:19.971733 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:19.971706 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:19.971733 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:19.971735 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:19.972263 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:19.971809 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:19.972263 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:19.971850 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:19.972263 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:19.971925 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:19.972263 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:19.971994 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:21.972356 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.972299 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:21.972841 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.972422 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:21.972841 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.972516 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:21.972841 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:21.972554 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:21.972841 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:21.972672 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:21.972841 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:21.972750 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:21.996814 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.996787 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:06:21.996987 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.996893 2570 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 02:06:21.997720 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:21.997697 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:06:23.907703 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:23.907675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:23.908131 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:23.907724 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:23.908131 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.907817 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:06:23.908131 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.907815 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 02:06:23.908131 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.907877 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs podName:9082f258-87ca-4149-9e1e-69e87f0b2cb6 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:55.907860317 +0000 UTC m=+65.439632678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs") pod "network-metrics-daemon-5h9d5" (UID: "9082f258-87ca-4149-9e1e-69e87f0b2cb6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 02:06:23.908131 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.907896 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret podName:c66eb7a2-b11d-422a-b75c-4dc43e4efe5d nodeName:}" failed. No retries permitted until 2026-11-04 02:06:55.907886684 +0000 UTC m=+65.439659033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret") pod "global-pull-secret-syncer-tjxq6" (UID: "c66eb7a2-b11d-422a-b75c-4dc43e4efe5d") : object "kube-system"/"original-pull-secret" not registered Nov 04 02:06:23.972512 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:23.972479 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:23.972512 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:23.972500 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:23.972725 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:23.972594 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:23.972725 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.972599 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:23.972725 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.972654 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:23.972838 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:23.972732 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:24.008069 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:24.008029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:24.008245 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:24.008180 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 02:06:24.008245 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:24.008198 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 02:06:24.008245 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:24.008210 2570 projected.go:196] Error preparing data for projected volume kube-api-access-l4jzz for pod openshift-network-diagnostics/network-check-target-56kwr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:06:24.008440 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:24.008293 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz podName:c89b664a-dd5d-4631-bbae-d197c38825e2 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:56.00827476 +0000 UTC m=+65.540047124 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4jzz" (UniqueName: "kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz") pod "network-check-target-56kwr" (UID: "c89b664a-dd5d-4631-bbae-d197c38825e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 02:06:25.075769 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.075745 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjxq6"] Nov 04 02:06:25.076089 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.075843 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:25.076089 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:25.075928 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:25.079150 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.079121 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-56kwr"] Nov 04 02:06:25.079256 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.079223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:25.079365 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:25.079312 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:25.081217 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.081198 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5h9d5"] Nov 04 02:06:25.081321 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:25.081292 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:25.081435 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:25.081405 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:26.972346 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:26.972305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:26.972773 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:26.972310 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:26.972773 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:26.972456 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:26.972773 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:26.972466 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:26.972773 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:26.972536 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:26.972773 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:26.972602 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:28.975252 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:28.975223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:28.975753 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:28.975318 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56kwr" podUID="c89b664a-dd5d-4631-bbae-d197c38825e2" Nov 04 02:06:28.975753 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:28.975358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:28.975753 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:28.975488 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5h9d5" podUID="9082f258-87ca-4149-9e1e-69e87f0b2cb6" Nov 04 02:06:28.975753 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:28.975534 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:28.975753 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:06:28.975591 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjxq6" podUID="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" Nov 04 02:06:29.694672 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.694500 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeReady" Nov 04 02:06:29.694818 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.694757 2570 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 02:06:29.852855 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.851751 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mcj49"] Nov 04 02:06:29.922552 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.922509 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gr2cj"] Nov 04 02:06:29.922722 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.922601 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:29.939711 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.939687 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7fl4d"] Nov 04 02:06:29.939816 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.939799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:29.953871 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.953851 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mcj49"] Nov 04 02:06:29.953943 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.953883 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gr2cj"] Nov 04 02:06:29.953985 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.953969 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:29.976105 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.976083 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 02:06:29.976396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.976177 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 02:06:29.976396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.976187 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zd6qr\"" Nov 04 02:06:29.976396 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.976189 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 02:06:29.983519 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.983501 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7fl4d"] Nov 04 02:06:29.997977 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:29.997960 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 02:06:30.023107 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.023088 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 02:06:30.023222 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.023208 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 02:06:30.023277 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.023268 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 02:06:30.055065 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.055158 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st24x\" (UniqueName: \"kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.055158 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055090 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.055241 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055192 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjl6g\" (UniqueName: \"kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.055241 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.055241 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055234 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.055415 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055300 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.055415 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.055415 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055401 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.055517 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055425 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwhts\" (UniqueName: \"kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.055517 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.055499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.080847 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.080829 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 02:06:30.080847 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.080842 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 02:06:30.080847 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.080831 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7wvp\"" Nov 04 02:06:30.080978 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.080833 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzhlz\"" Nov 04 02:06:30.156470 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156403 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st24x\" (UniqueName: \"kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.156470 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjl6g\" (UniqueName: \"kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156507 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156530 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.156649 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhts\" (UniqueName: \"kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.156962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.156962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156686 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.156962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.156962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.156939 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.157197 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.157056 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.157392 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.157375 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.157469 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.157409 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.160964 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.160941 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.160964 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.160953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.161089 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.160949 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.216873 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.216843 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhts\" (UniqueName: \"kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.216983 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.216942 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjl6g\" (UniqueName: \"kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.217066 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.217045 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st24x\" (UniqueName: \"kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.248875 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.248846 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:06:30.274010 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.273978 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:30.301987 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.301956 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:06:30.316154 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.316123 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:06:30.328969 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.327496 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.364349 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.364306 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:06:30.380452 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.380406 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 02:06:30.380644 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.380487 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jsvqb\"" Nov 04 02:06:30.380644 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.380508 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 02:06:30.380644 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.380573 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 02:06:30.380826 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.380736 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 02:06:30.381422 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.381353 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 02:06:30.381422 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.381378 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 02:06:30.381422 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.381380 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 02:06:30.458285 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458458 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458348 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458458 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458377 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458458 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458398 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458458 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458420 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458458 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.458618 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.458522 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559129 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559097 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559129 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559323 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559150 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559323 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559184 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559323 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559323 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559519 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.559575 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559541 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.560003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559977 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.560108 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.559986 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.560295 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.560274 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.561897 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.561879 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.562002 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.561986 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.620005 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.619974 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.650898 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.650865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:30.759436 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.759410 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mcj49"] Nov 04 02:06:30.763191 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:30.763165 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3889db9a_bb9d_422e_a8a6_ffda0e64d867.slice/crio-85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439 WatchSource:0}: Error finding container 85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439: Status 404 returned error can't find the container with id 85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439 Nov 04 02:06:30.763452 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.763370 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gr2cj"] Nov 04 02:06:30.763978 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.763959 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7fl4d"] Nov 04 02:06:30.766080 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:30.766057 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b193ea_8136_4bb9_b98b_a3930e51f99c.slice/crio-97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92 WatchSource:0}: Error finding container 97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92: Status 404 returned error can't find the container with id 97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92 Nov 04 02:06:30.766987 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:30.766965 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced39f9f_ae73_431b_aeb4_bb0b2ae2b218.slice/crio-dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c WatchSource:0}: Error finding container dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c: Status 404 returned error can't find the container with id dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c Nov 04 02:06:30.975485 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.975177 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:30.975609 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.975187 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:30.975609 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:30.975194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:31.027819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.027789 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.027867 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.027917 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.028011 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h9jgb\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.028017 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8fdm8\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.028040 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ghp2x\"" Nov 04 02:06:31.028463 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.028047 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 02:06:31.045213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.045196 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:06:31.048028 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:31.048005 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3adefd_eb97_468e_912b_fd79b7fa7ee2.slice/crio-1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a WatchSource:0}: Error finding container 1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a: Status 404 returned error can't find the container with id 1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a Nov 04 02:06:31.196549 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.196513 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c76cc68f9-dcjvf" event={"ID":"ed3adefd-eb97-468e-912b-fd79b7fa7ee2","Type":"ContainerStarted","Data":"1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a"} Nov 04 02:06:31.197922 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.197892 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7fl4d" event={"ID":"70b193ea-8136-4bb9-b98b-a3930e51f99c","Type":"ContainerStarted","Data":"c619b9635a00646c1d06f6874a4c792565653ac8b0b737cbf56ed61c7cda93f5"} Nov 04 02:06:31.198034 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.197941 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7fl4d" event={"ID":"70b193ea-8136-4bb9-b98b-a3930e51f99c","Type":"ContainerStarted","Data":"97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92"} Nov 04 02:06:31.199033 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.199011 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gr2cj" event={"ID":"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218","Type":"ContainerStarted","Data":"dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c"} Nov 04 02:06:31.200062 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:31.200042 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mcj49" event={"ID":"3889db9a-bb9d-422e-a8a6-ffda0e64d867","Type":"ContainerStarted","Data":"85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439"} Nov 04 02:06:33.209031 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.208983 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7fl4d" event={"ID":"70b193ea-8136-4bb9-b98b-a3930e51f99c","Type":"ContainerStarted","Data":"1b0ec85792dd3e2c2e9cc804900ae0da407dc4c348172a74e2126b54806b9042"} Nov 04 02:06:33.473812 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.473731 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv"] Nov 04 02:06:33.491587 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.491563 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.535751 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.535719 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv"] Nov 04 02:06:33.581864 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.581826 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.582043 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.581873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plqx\" (UniqueName: \"kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.582105 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.582037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.582105 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.582078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.584615 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.584473 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 02:06:33.584615 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.584507 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 02:06:33.584615 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.584588 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sd8ds\"" Nov 04 02:06:33.593167 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.593143 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 02:06:33.593271 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.593180 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 02:06:33.593271 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.593180 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 02:06:33.635381 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.635353 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-shpnc"] Nov 04 02:06:33.650702 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.650679 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4"] Nov 04 02:06:33.650847 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.650794 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.668874 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.668853 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.682885 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.682861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.683012 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.682899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.683095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.683036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.683245 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.683120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plqx\" (UniqueName: \"kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.683579 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.683560 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.686880 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.686852 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.687202 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.687180 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.692207 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.692185 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4"] Nov 04 02:06:33.748128 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748057 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 02:06:33.748128 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748073 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 02:06:33.748128 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748115 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xlwps\"" Nov 04 02:06:33.748128 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748123 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 02:06:33.748482 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748200 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 02:06:33.748482 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748209 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qgk7m\"" Nov 04 02:06:33.748482 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748262 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 02:06:33.748482 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.748418 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 02:06:33.757276 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.757248 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plqx\" (UniqueName: \"kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.784152 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784307 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784173 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784307 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784465 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.784465 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784406 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.784465 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784443 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.784588 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.784588 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784511 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.784588 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784614 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7g5l\" (UniqueName: \"kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784655 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784816 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784683 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784816 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.784816 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.784734 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rp4\" (UniqueName: \"kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.847279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.847245 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:06:33.886666 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.886827 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.886827 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886732 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rp4\" (UniqueName: \"kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.886827 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.886967 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.886967 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.886967 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.886967 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.886978 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887032 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887061 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887100 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887142 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7g5l\" (UniqueName: \"kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887454 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887153 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887454 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887189 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887454 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887249 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887454 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887322 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887643 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887579 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.887730 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887708 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.887822 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.887805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.888095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.888077 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.888376 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.888348 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.889412 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.888554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.890917 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.890886 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.891197 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.891177 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.891268 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.891200 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.892737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.892707 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.949356 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.949281 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rp4\" (UniqueName: \"kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:33.991675 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.991607 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7g5l\" (UniqueName: \"kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:33.994537 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:33.994259 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:06:34.028045 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:34.028016 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d57b05f_6e23_42fb_b3ff_25479d15b6ae.slice/crio-2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae WatchSource:0}: Error finding container 2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae: Status 404 returned error can't find the container with id 2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae Nov 04 02:06:34.035023 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:34.035003 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:06:34.216195 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:34.215960 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gr2cj" event={"ID":"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218","Type":"ContainerStarted","Data":"46b760d55f37686f7c3304a1784d01eca9d2598193ae63257a1cb1b779a84963"} Nov 04 02:06:34.217525 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:34.217351 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-shpnc" event={"ID":"9d57b05f-6e23-42fb-b3ff-25479d15b6ae","Type":"ContainerStarted","Data":"2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae"} Nov 04 02:06:34.355635 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:34.355607 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv"] Nov 04 02:06:34.358747 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:34.358709 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf92aa8_a824_4089_8fa8_9005b529346f.slice/crio-85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f WatchSource:0}: Error finding container 85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f: Status 404 returned error can't find the container with id 85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f Nov 04 02:06:34.468088 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:34.468059 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4"] Nov 04 02:06:34.472999 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:34.472428 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0129d90_8271_46a8_b92b_e179632baf01.slice/crio-1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25 WatchSource:0}: Error finding container 1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25: Status 404 returned error can't find the container with id 1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25 Nov 04 02:06:35.225049 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.224723 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" event={"ID":"caf92aa8-a824-4089-8fa8-9005b529346f","Type":"ContainerStarted","Data":"533e3b2e19978d7e7fa0b262b9b30116592ca37d0ff81aae611bac7324169274"} Nov 04 02:06:35.225049 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.224801 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" event={"ID":"caf92aa8-a824-4089-8fa8-9005b529346f","Type":"ContainerStarted","Data":"37190a47b481eac6d739644374822a15e7e9d9876411e8097267bc5e004f944a"} Nov 04 02:06:35.225049 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.224845 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" event={"ID":"caf92aa8-a824-4089-8fa8-9005b529346f","Type":"ContainerStarted","Data":"85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f"} Nov 04 02:06:35.226974 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.226940 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gr2cj" event={"ID":"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218","Type":"ContainerStarted","Data":"7789d9757566e768579d54320ad736e071d389e53d2bbcd0c7aa20057afc96c5"} Nov 04 02:06:35.227111 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.227057 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:35.228742 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.228720 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mcj49" event={"ID":"3889db9a-bb9d-422e-a8a6-ffda0e64d867","Type":"ContainerStarted","Data":"4a0c9ae19ba683928a5aa6e7dd2c1d420ec1dd5b5d22ab4e0e53c251c527a904"} Nov 04 02:06:35.229913 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.229890 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" event={"ID":"d0129d90-8271-46a8-b92b-e179632baf01","Type":"ContainerStarted","Data":"1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25"} Nov 04 02:06:35.346446 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.346181 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-gr2cj" podStartSLOduration=3.179477334 podStartE2EDuration="6.346162682s" totalImagesPullingTime="3.166685348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:35.344815774 +0000 UTC m=+44.876588154" watchObservedRunningTime="2026-11-04 02:06:35.346162682 +0000 UTC m=+44.877935053" Nov 04 02:06:35.460535 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.460483 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mcj49" podStartSLOduration=2.154273169 podStartE2EDuration="6.460464147s" totalImagesPullingTime="4.306190978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:35.459961156 +0000 UTC m=+44.991733526" watchObservedRunningTime="2026-11-04 02:06:35.460464147 +0000 UTC m=+44.992236518" Nov 04 02:06:35.879120 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.879082 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:06:35.893481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.893457 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:35.948807 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.948780 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 02:06:35.948952 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.948834 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 02:06:35.948952 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.948876 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 02:06:35.948952 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.948930 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 02:06:35.949393 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.949364 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fjjw2\"" Nov 04 02:06:35.949527 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.949453 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 02:06:35.949527 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.949520 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 02:06:35.949644 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.949591 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 02:06:35.950444 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:35.950425 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:06:36.003591 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003591 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003596 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003799 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003629 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003799 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003742 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003799 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003918 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003918 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003918 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.003918 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003912 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.004108 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.004108 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.003992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.004108 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.004025 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.004108 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.004085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.048247 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.048218 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 02:06:36.048715 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.048682 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 02:06:36.105171 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105206 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105236 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105293 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105391 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105611 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.105922 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.105606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.107236 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.106731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.108286 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.107947 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.108286 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.108029 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.113438 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.113375 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.113531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.113464 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.115304 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.114773 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.115304 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.114816 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.116184 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.116037 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.116184 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.116145 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.116648 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.116492 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.116648 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.116619 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.117772 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.117748 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.169359 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.169251 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.238672 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.238637 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c76cc68f9-dcjvf" event={"ID":"ed3adefd-eb97-468e-912b-fd79b7fa7ee2","Type":"ContainerStarted","Data":"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a"} Nov 04 02:06:36.241658 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.241598 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-shpnc" event={"ID":"9d57b05f-6e23-42fb-b3ff-25479d15b6ae","Type":"ContainerStarted","Data":"56112a0cb724e296eb374c27e2bf3a39cf5db71872bd856a15716183cb062825"} Nov 04 02:06:36.248737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.248713 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7fl4d" event={"ID":"70b193ea-8136-4bb9-b98b-a3930e51f99c","Type":"ContainerStarted","Data":"cf136c0450a3bf90d3c8b0eb80047b1550318bd244b75cdb7c377cd45ac53f12"} Nov 04 02:06:36.316489 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.316419 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:06:36.316690 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.316667 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg"] Nov 04 02:06:36.323732 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.323043 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.360264 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.359612 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c76cc68f9-dcjvf" podStartSLOduration=2.100109902 podStartE2EDuration="6.359592845s" totalImagesPullingTime="4.259482943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:36.358548356 +0000 UTC m=+45.890320726" watchObservedRunningTime="2026-11-04 02:06:36.359592845 +0000 UTC m=+45.891365216" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375643 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375715 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375723 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7kg86\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375654 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375726 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375789 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2iitb07vi68ac\"" Nov 04 02:06:36.375819 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.375792 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 02:06:36.381523 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.381405 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg"] Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409042 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzdhz\" (UniqueName: \"kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409214 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409409 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409577 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.413735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.409631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.510885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511253 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511253 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511152 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511253 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511253 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzdhz\" (UniqueName: \"kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511253 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511246 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511475 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511475 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.511475 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.511318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.513010 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.512511 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.513010 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.512972 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.515610 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.515587 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.515942 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.515918 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.516152 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.516132 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.516710 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.516686 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.516809 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.516731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.518217 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.518169 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.572537 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.572509 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzdhz\" (UniqueName: \"kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.608780 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.608710 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7fl4d" podStartSLOduration=2.504911976 podStartE2EDuration="7.608680836s" totalImagesPullingTime="5.10376886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:06:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:36.608120612 +0000 UTC m=+46.139892983" watchObservedRunningTime="2026-11-04 02:06:36.608680836 +0000 UTC m=+46.140453205" Nov 04 02:06:36.736636 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.736549 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:36.803241 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:36.803211 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:06:36.850185 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:36.850150 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b751c1_c3f8_4b3e_aee3_5d3b42cb4d37.slice/crio-752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0 WatchSource:0}: Error finding container 752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0: Status 404 returned error can't find the container with id 752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0 Nov 04 02:06:37.128443 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.127966 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg"] Nov 04 02:06:37.130274 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:37.130249 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e90242_f97c_45f4_abd7_4493c8d9cc42.slice/crio-9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264 WatchSource:0}: Error finding container 9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264: Status 404 returned error can't find the container with id 9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264 Nov 04 02:06:37.258072 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.257936 2570 generic.go:356] "Generic (PLEG): container finished" podID="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" containerID="56112a0cb724e296eb374c27e2bf3a39cf5db71872bd856a15716183cb062825" exitCode=0 Nov 04 02:06:37.258072 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.258034 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-shpnc" event={"ID":"9d57b05f-6e23-42fb-b3ff-25479d15b6ae","Type":"ContainerDied","Data":"56112a0cb724e296eb374c27e2bf3a39cf5db71872bd856a15716183cb062825"} Nov 04 02:06:37.260515 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.260490 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" event={"ID":"caf92aa8-a824-4089-8fa8-9005b529346f","Type":"ContainerStarted","Data":"b4cec85c55df553d856b50bf7b24f86d1eb7aaa2dea307277b33a4822482d143"} Nov 04 02:06:37.261692 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.261671 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0"} Nov 04 02:06:37.262723 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.262700 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264"} Nov 04 02:06:37.264687 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.264662 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" event={"ID":"d0129d90-8271-46a8-b92b-e179632baf01","Type":"ContainerStarted","Data":"6ff981aa1e8a731114e343f95ace15d9e4b55d186f0d8404f9e4cf72183f6a22"} Nov 04 02:06:37.264788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.264716 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" event={"ID":"d0129d90-8271-46a8-b92b-e179632baf01","Type":"ContainerStarted","Data":"34888093f05653dbb59c390d7f90970f6ecd6b8689de46e9cbb9c1c328724288"} Nov 04 02:06:37.264788 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.264746 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" event={"ID":"d0129d90-8271-46a8-b92b-e179632baf01","Type":"ContainerStarted","Data":"91cdd7325090c0525e08dc4f7209ad8939d6ea853e2c288523297e931736ebe7"} Nov 04 02:06:37.493619 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.493416 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" podStartSLOduration=2.241443847 podStartE2EDuration="4.493402785s" totalImagesPullingTime="2.251958938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:37.492110664 +0000 UTC m=+47.023883056" watchObservedRunningTime="2026-11-04 02:06:37.493402785 +0000 UTC m=+47.025175154" Nov 04 02:06:37.610233 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:37.610173 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" podStartSLOduration=2.3416227640000002 podStartE2EDuration="4.610159235s" totalImagesPullingTime="2.268536471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:37.609320399 +0000 UTC m=+47.141092767" watchObservedRunningTime="2026-11-04 02:06:37.610159235 +0000 UTC m=+47.141931603" Nov 04 02:06:38.270975 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.270937 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-shpnc" event={"ID":"9d57b05f-6e23-42fb-b3ff-25479d15b6ae","Type":"ContainerStarted","Data":"369a66014706d2f47d90d28d1663c062afd16d27a4a8ecc7b443df188767ceaa"} Nov 04 02:06:38.271390 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.270999 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-shpnc" event={"ID":"9d57b05f-6e23-42fb-b3ff-25479d15b6ae","Type":"ContainerStarted","Data":"9114a445948695d1c1f57241cb76496a570c37102e1dcc44c148e8b38f60e3bd"} Nov 04 02:06:38.321803 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.321762 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr"] Nov 04 02:06:38.344468 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.344444 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.382640 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.382608 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr"] Nov 04 02:06:38.390863 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.390820 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-shpnc" podStartSLOduration=3.220737404 podStartE2EDuration="5.390804841s" totalImagesPullingTime="2.078368274s" totalInitContainerRuntime="91.699163ms" isStatefulPod=true podCreationTimestamp="2026-11-04 02:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:38.390153879 +0000 UTC m=+47.921926247" watchObservedRunningTime="2026-11-04 02:06:38.390804841 +0000 UTC m=+47.922577210" Nov 04 02:06:38.403209 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.403188 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 02:06:38.403557 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.403509 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6hm7q\"" Nov 04 02:06:38.528106 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.528029 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-db9bb74dd-5bnpg"] Nov 04 02:06:38.532167 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.532142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95zw\" (UniqueName: \"kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.532285 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.532238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.534882 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.534866 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.590659 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.590630 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 02:06:38.590897 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.590751 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 02:06:38.590985 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.590920 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 02:06:38.590985 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.590941 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-w9wdz\"" Nov 04 02:06:38.591085 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.591066 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 02:06:38.591212 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.591195 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8lp1mckvjcva\"" Nov 04 02:06:38.592403 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.592382 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db9bb74dd-5bnpg"] Nov 04 02:06:38.633121 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633290 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633135 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633371 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633289 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.633371 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633479 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633526 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633503 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633573 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633542 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjq8c\" (UniqueName: \"kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633629 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.633677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.633625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95zw\" (UniqueName: \"kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.637492 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.637462 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.698481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.698450 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95zw\" (UniqueName: \"kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:38.734823 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.734823 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734825 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjq8c\" (UniqueName: \"kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734922 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735059 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.734970 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735416 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.735263 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.735890 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.735836 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.736213 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.736192 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.737955 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.737930 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.738083 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.738066 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.738377 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.738353 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.799157 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.799133 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjq8c\" (UniqueName: \"kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.851532 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.851501 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:38.968015 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:38.967987 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:39.276786 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.276750 2570 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4" exitCode=0 Nov 04 02:06:39.277501 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.277473 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4"} Nov 04 02:06:39.290773 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.290626 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db9bb74dd-5bnpg"] Nov 04 02:06:39.293820 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:39.293794 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a78f67_2bc4_4d66_b09a_e93063641f64.slice/crio-72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c WatchSource:0}: Error finding container 72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c: Status 404 returned error can't find the container with id 72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c Nov 04 02:06:39.341953 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.341869 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp"] Nov 04 02:06:39.344646 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.344624 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.399451 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.399405 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Nov 04 02:06:39.399593 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.399460 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Nov 04 02:06:39.401466 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.401440 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr"] Nov 04 02:06:39.405652 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:39.405629 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825cfc8b_20b7_481c_beaa_46a414be370a.slice/crio-f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5 WatchSource:0}: Error finding container f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5: Status 404 returned error can't find the container with id f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5 Nov 04 02:06:39.408393 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.408304 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp"] Nov 04 02:06:39.434945 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.434920 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Nov 04 02:06:39.441858 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.441824 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.441965 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.441864 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.441965 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.441889 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.442075 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.441991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.442189 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.442168 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.442383 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.442362 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.442708 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.442686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d78kv\" (UniqueName: \"kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.442811 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.442745 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.462481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.462458 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Nov 04 02:06:39.500474 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.500448 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Nov 04 02:06:39.500474 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.500462 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lnjv7\"" Nov 04 02:06:39.500474 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.500450 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Nov 04 02:06:39.543517 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543671 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543671 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543783 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543783 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543865 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543897 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d78kv\" (UniqueName: \"kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.543932 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.543904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.544278 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.544254 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.544523 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.544500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.545055 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.545032 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.546490 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.546469 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.546685 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.546668 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.546760 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.546703 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.546828 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.546812 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.605241 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.605172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78kv\" (UniqueName: \"kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:39.657832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:39.657805 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:06:40.094708 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.094662 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp"] Nov 04 02:06:40.097361 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:40.097305 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7014e7c0_9d7d_4306_8e68_98437feeaf4e.slice/crio-5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a WatchSource:0}: Error finding container 5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a: Status 404 returned error can't find the container with id 5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a Nov 04 02:06:40.282520 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.282478 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"ecb24fd48bbdac612cf5ee37d3ba0e6829b785dc5e1fab647aaff939ec27c4a9"} Nov 04 02:06:40.282911 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.282558 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"83ce0f6c157f291d45838e4cbed82a113684678841d54de2fa37dbf2d9b6930d"} Nov 04 02:06:40.282911 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.282593 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"003576d89bc49bb98e643695dea911252d83a113abf2cbb6a6cd3bb4603492a1"} Nov 04 02:06:40.283764 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.283734 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" event={"ID":"7014e7c0-9d7d-4306-8e68-98437feeaf4e","Type":"ContainerStarted","Data":"5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a"} Nov 04 02:06:40.285013 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.284988 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" event={"ID":"825cfc8b-20b7-481c-beaa-46a414be370a","Type":"ContainerStarted","Data":"f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5"} Nov 04 02:06:40.286473 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.286453 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" event={"ID":"40a78f67-2bc4-4d66-b09a-e93063641f64","Type":"ContainerStarted","Data":"72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c"} Nov 04 02:06:40.651606 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.651536 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:40.651832 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.651809 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:06:40.652645 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.652623 2570 patch_prober.go:29] interesting pod/console-7c76cc68f9-dcjvf container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused" start-of-body= Nov 04 02:06:40.652737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:40.652661 2570 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-console/console-7c76cc68f9-dcjvf" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerName="console" probeResult="failure" output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused" Nov 04 02:06:41.394719 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.394627 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:06:41.396958 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.396936 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.501807 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.501771 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:06:41.502034 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.502016 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 02:06:41.565494 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565503 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565527 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565644 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565677 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565664 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.565899 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.565725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.666391 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.666320 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.666936 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.666897 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.667036 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.666984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.667076 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.667055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.667125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.667113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.667758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.667736 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.668513 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.668485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.668779 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.668758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.668962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.668801 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.668962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.668848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.668962 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.668877 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.670029 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.670004 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.670983 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.670647 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.670983 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.670942 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.671972 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.671929 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:41.732251 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:41.732222 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:42.010675 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.010513 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:42.297116 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.297087 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" event={"ID":"825cfc8b-20b7-481c-beaa-46a414be370a","Type":"ContainerStarted","Data":"fa9c0aeae81b3ac6380c6fba84a0e531582a31fb9c026b558ba3c3ce30edfded"} Nov 04 02:06:42.297985 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.297966 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:42.299471 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.299443 2570 patch_prober.go:29] interesting pod/monitoring-plugin-77f4f76594-fw6nr container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.133.0.14:9443/health\": dial tcp 10.133.0.14:9443: connect: connection refused" start-of-body= Nov 04 02:06:42.299605 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.299489 2570 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" podUID="825cfc8b-20b7-481c-beaa-46a414be370a" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.133.0.14:9443/health\": dial tcp 10.133.0.14:9443: connect: connection refused" Nov 04 02:06:42.300261 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.300234 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" event={"ID":"40a78f67-2bc4-4d66-b09a-e93063641f64","Type":"ContainerStarted","Data":"ca43cb08bacef72e903d9609078f9fa95b2f55a354ca76ffd3e128d2a6a0c4ca"} Nov 04 02:06:42.306180 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.306155 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"673f8403d10c01e3737547eab6751000b7181555bb501b9a5d5900a0b0fbbaee"} Nov 04 02:06:42.306285 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.306221 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"2116ff50a455f0407335cc2c8c9b11bbc201a3e71cfdb197f8239d4a4dbbeed0"} Nov 04 02:06:42.306285 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.306260 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" event={"ID":"e2e90242-f97c-45f4-abd7-4493c8d9cc42","Type":"ContainerStarted","Data":"647bbf1f605e7c962c413c54508f0e7d22c4b7cd4104905816aea9b6bfc2058c"} Nov 04 02:06:42.306420 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.306364 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:42.395838 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.395805 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:06:42.411521 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.411471 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" podStartSLOduration=1.5931664840000002 podStartE2EDuration="4.411455966s" totalImagesPullingTime="2.818289482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:42.410692486 +0000 UTC m=+51.942464854" watchObservedRunningTime="2026-11-04 02:06:42.411455966 +0000 UTC m=+51.943228337" Nov 04 02:06:42.426382 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:42.426357 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50529fc_30c2_4a6f_83db_f90819ad9a94.slice/crio-ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1 WatchSource:0}: Error finding container ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1: Status 404 returned error can't find the container with id ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1 Nov 04 02:06:42.660521 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.660414 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" podStartSLOduration=2.224603554 podStartE2EDuration="4.660393202s" totalImagesPullingTime="2.435789648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:42.533396661 +0000 UTC m=+52.065169032" watchObservedRunningTime="2026-11-04 02:06:42.660393202 +0000 UTC m=+52.192165571" Nov 04 02:06:42.661151 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.661133 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:06:42.664135 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.664114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682317 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682496 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682630 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682679 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682670 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682916 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682916 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682916 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.682916 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682888 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.682932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683029 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683095 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683216 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683153 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683216 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683311 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683263 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683393 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683443 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683393 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.683443 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.683421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.687264 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.687222 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" podStartSLOduration=2.414885018 podStartE2EDuration="6.687210099s" totalImagesPullingTime="4.272325081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:42.686454437 +0000 UTC m=+52.218226807" watchObservedRunningTime="2026-11-04 02:06:42.687210099 +0000 UTC m=+52.218982491" Nov 04 02:06:42.716375 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.716311 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 02:06:42.716564 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.716481 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 02:06:42.716653 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.716577 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 02:06:42.716858 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.716839 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fh89s\"" Nov 04 02:06:42.717125 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717106 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 02:06:42.717222 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717209 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 02:06:42.717651 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717631 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717631 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717686 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717685 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717688 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717743 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 02:06:42.717758 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717756 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-26iig1f3jpl7f\"" Nov 04 02:06:42.717936 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717760 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 02:06:42.717936 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.717803 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 02:06:42.766114 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.766088 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 02:06:42.771379 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.771358 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:06:42.783892 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.783865 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.783912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.783957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.783999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784030 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784057 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784090 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784119 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784150 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784240 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784209 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784268 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784370 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784410 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784486 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784523 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784547 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.784626 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784589 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.785029 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.784932 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.786535 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.786504 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.787449 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.787350 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.788088 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.787592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.788088 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.787618 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.788389 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.788303 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.788840 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.788815 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.789644 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.789618 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.792374 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.789974 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.792374 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.790154 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.792374 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.790577 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.792374 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.791268 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.792374 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.791706 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.794121 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.794049 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.794121 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.794076 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.794830 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.794782 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.795543 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.795489 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.796223 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.796203 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.796591 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.796570 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.916924 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.916782 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:42.995733 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:42.995689 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:43.315305 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.315265 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8cf5d56b-xqwgh" event={"ID":"a50529fc-30c2-4a6f-83db-f90819ad9a94","Type":"ContainerStarted","Data":"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1"} Nov 04 02:06:43.315493 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.315376 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8cf5d56b-xqwgh" event={"ID":"a50529fc-30c2-4a6f-83db-f90819ad9a94","Type":"ContainerStarted","Data":"ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1"} Nov 04 02:06:43.317303 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.317268 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" event={"ID":"7014e7c0-9d7d-4306-8e68-98437feeaf4e","Type":"ContainerStarted","Data":"c5af8d2541bfc1e9dd2cd89264045369ae9c48aec8e41eb9c696ded7e650ab05"} Nov 04 02:06:43.317464 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.317324 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" event={"ID":"7014e7c0-9d7d-4306-8e68-98437feeaf4e","Type":"ContainerStarted","Data":"b46ac0f097e7f9aa4e4d199c36036bb6a95573dd0ddf9230b00f687882344318"} Nov 04 02:06:43.317464 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.317372 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" event={"ID":"7014e7c0-9d7d-4306-8e68-98437feeaf4e","Type":"ContainerStarted","Data":"39bed0aca6607aba0c70d91455da421fef48b5bfa6a9070c3960abb5331974ee"} Nov 04 02:06:43.322044 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.322022 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:06:43.428826 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.428745 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:06:43.432914 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.432871 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-d8cf5d56b-xqwgh" podStartSLOduration=2.432856341 podStartE2EDuration="2.432856341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:43.431973315 +0000 UTC m=+52.963745686" watchObservedRunningTime="2026-11-04 02:06:43.432856341 +0000 UTC m=+52.964628709" Nov 04 02:06:43.443694 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:43.443668 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d93c1b_1835_43d7_88cf_371913654014.slice/crio-b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9 WatchSource:0}: Error finding container b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9: Status 404 returned error can't find the container with id b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9 Nov 04 02:06:43.600905 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:43.600812 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" podStartSLOduration=1.972087305 podStartE2EDuration="4.600797994s" totalImagesPullingTime="2.628710689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:43.599493837 +0000 UTC m=+53.131266206" watchObservedRunningTime="2026-11-04 02:06:43.600797994 +0000 UTC m=+53.132570363" Nov 04 02:06:44.321920 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.321884 2570 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" exitCode=0 Nov 04 02:06:44.322166 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.321970 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} Nov 04 02:06:44.322166 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.322042 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9"} Nov 04 02:06:44.325449 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325426 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229"} Nov 04 02:06:44.325589 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325490 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0"} Nov 04 02:06:44.325589 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325520 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68"} Nov 04 02:06:44.325589 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325544 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde"} Nov 04 02:06:44.325589 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325575 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888"} Nov 04 02:06:44.325791 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.325605 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerStarted","Data":"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da"} Nov 04 02:06:44.657806 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:44.657708 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.738625852 podStartE2EDuration="9.65769324s" totalImagesPullingTime="5.901126536s" totalInitContainerRuntime="17.940852ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:44.655731976 +0000 UTC m=+54.187504384" watchObservedRunningTime="2026-11-04 02:06:44.65769324 +0000 UTC m=+54.189465608" Nov 04 02:06:45.252476 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:45.252451 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gr2cj" Nov 04 02:06:47.176942 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:47.176917 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:06:48.323502 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:48.323473 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:06:49.809979 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.809938 2570 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:06:49.853060 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.853027 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:06:49.855309 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.855282 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.908313 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.908277 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:06:49.956746 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.956928 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956783 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.956928 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956897 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.956928 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.957079 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.957079 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.956999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.957079 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.957070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:49.957188 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:49.957089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058064 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058064 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058316 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058607 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.058607 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058519 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.059014 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.058993 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.059122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.059017 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.059122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.059089 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.059543 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.059523 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.061132 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.061073 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.061371 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.061347 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.122111 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.122086 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.168214 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.168166 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:06:50.352021 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.351993 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} Nov 04 02:06:50.548197 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:50.548171 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:06:50.553362 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:50.550109 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c21249_e6ab_42b6_9391_a9f4ad402b8a.slice/crio-8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1 WatchSource:0}: Error finding container 8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1: Status 404 returned error can't find the container with id 8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1 Nov 04 02:06:51.358684 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.358643 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} Nov 04 02:06:51.359122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.358727 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} Nov 04 02:06:51.359122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.358767 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} Nov 04 02:06:51.359122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.358809 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} Nov 04 02:06:51.359122 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.358843 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerStarted","Data":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} Nov 04 02:06:51.360321 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.360293 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75dd9456cc-2tsv2" event={"ID":"01c21249-e6ab-42b6-9391-a9f4ad402b8a","Type":"ContainerStarted","Data":"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97"} Nov 04 02:06:51.360456 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.360350 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75dd9456cc-2tsv2" event={"ID":"01c21249-e6ab-42b6-9391-a9f4ad402b8a","Type":"ContainerStarted","Data":"8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1"} Nov 04 02:06:51.539264 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.539207 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.585023378 podStartE2EDuration="9.539192855s" totalImagesPullingTime="5.932542317s" totalInitContainerRuntime="21.62716ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:51.538426137 +0000 UTC m=+61.070198507" watchObservedRunningTime="2026-11-04 02:06:51.539192855 +0000 UTC m=+61.070965224" Nov 04 02:06:51.656637 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:51.656539 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-75dd9456cc-2tsv2" podStartSLOduration=2.656524811 podStartE2EDuration="2.656524811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:51.655312893 +0000 UTC m=+61.187085263" watchObservedRunningTime="2026-11-04 02:06:51.656524811 +0000 UTC m=+61.188297180" Nov 04 02:06:52.010993 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:52.010908 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:52.010993 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:52.010963 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:52.015692 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:52.015668 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:52.367712 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:52.367686 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:06:52.996425 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:52.996399 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:06:55.916712 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:55.916679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:55.917180 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:55.916726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:55.968188 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:55.968158 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 02:06:55.979688 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:55.979661 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:56.017689 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.017663 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 02:06:56.017876 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.017855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:56.029943 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.029916 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:56.069355 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.069311 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 02:06:56.129746 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.129721 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 02:06:56.141214 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.141188 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:56.254618 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.254548 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8fdm8\"" Nov 04 02:06:56.263705 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.263686 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:06:56.285897 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.285874 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h9jgb\"" Nov 04 02:06:56.294937 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.294910 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:06:56.300003 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.299977 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ghp2x\"" Nov 04 02:06:56.309185 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.309156 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:06:56.640189 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.640158 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-56kwr"] Nov 04 02:06:56.674032 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:56.674002 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89b664a_dd5d_4631_bbae_d197c38825e2.slice/crio-29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d WatchSource:0}: Error finding container 29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d: Status 404 returned error can't find the container with id 29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d Nov 04 02:06:56.690321 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.690299 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5h9d5"] Nov 04 02:06:56.691886 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:56.691864 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9082f258_87ca_4149_9e1e_69e87f0b2cb6.slice/crio-eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264 WatchSource:0}: Error finding container eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264: Status 404 returned error can't find the container with id eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264 Nov 04 02:06:56.737873 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:56.737855 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjxq6"] Nov 04 02:06:56.740380 ip-10-0-138-45 kubenswrapper[2570]: W1104 02:06:56.740357 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66eb7a2_b11d_422a_b75c_4dc43e4efe5d.slice/crio-110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4 WatchSource:0}: Error finding container 110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4: Status 404 returned error can't find the container with id 110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4 Nov 04 02:06:57.385165 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:57.385129 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5h9d5" event={"ID":"9082f258-87ca-4149-9e1e-69e87f0b2cb6","Type":"ContainerStarted","Data":"eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264"} Nov 04 02:06:57.386265 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:57.386245 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-56kwr" event={"ID":"c89b664a-dd5d-4631-bbae-d197c38825e2","Type":"ContainerStarted","Data":"29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d"} Nov 04 02:06:57.387207 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:57.387185 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tjxq6" event={"ID":"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d","Type":"ContainerStarted","Data":"110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4"} Nov 04 02:06:58.852239 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:58.852114 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:58.852804 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:58.852246 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:06:59.399886 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:59.399831 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5h9d5" event={"ID":"9082f258-87ca-4149-9e1e-69e87f0b2cb6","Type":"ContainerStarted","Data":"6970151cfab53266d7f259ead41843afcf09177642becb560c6c1c8aa0e276e3"} Nov 04 02:06:59.400069 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:06:59.399903 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5h9d5" event={"ID":"9082f258-87ca-4149-9e1e-69e87f0b2cb6","Type":"ContainerStarted","Data":"5ce68899b3075a1e00b8fc93f34dc6c2f4911e444535465bf7d77a8ac99e7747"} Nov 04 02:07:00.169279 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.169242 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:00.169735 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.169314 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:00.175481 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.175198 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:00.293511 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.293412 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5h9d5" podStartSLOduration=67.455942726 podStartE2EDuration="1m9.293391277s" totalImagesPullingTime="1.837448551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:59.520315377 +0000 UTC m=+69.052087746" watchObservedRunningTime="2026-11-04 02:07:00.293391277 +0000 UTC m=+69.825163647" Nov 04 02:07:00.352887 ip-10-0-138-45 kubenswrapper[2570]: E1104 02:07:00.352841 2570 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:07:00.410815 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.410779 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:00.559547 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:00.559523 2570 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:07:01.411531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:01.410980 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-56kwr" event={"ID":"c89b664a-dd5d-4631-bbae-d197c38825e2","Type":"ContainerStarted","Data":"8d212c73ab4ea61b6deff9fecf29c51d93ce68e0d17b69f89384a1350e6debd1"} Nov 04 02:07:01.411531 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:01.411059 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:07:01.525737 ip-10-0-138-45 kubenswrapper[2570]: I1104 02:07:01.525666 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-56kwr" podStartSLOduration=66.710265579 podStartE2EDuration="1m10.525626489s" totalImagesPullingTime="3.81536091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:01.525263922 +0000 UTC m=+71.057036293" watchObservedRunningTime="2026-11-04 02:07:01.525626489 +0000 UTC m=+71.057398859" Nov 04 02:07:02.228611 ip-10-0-138-45 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 02:07:02.242832 ip-10-0-138-45 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 02:07:02.243006 ip-10-0-138-45 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 02:07:02.243179 ip-10-0-138-45 systemd[1]: kubelet.service: Consumed 4.901s CPU time. Nov 04 02:07:02.264224 ip-10-0-138-45 systemd[1]: Starting Kubernetes Kubelet... Nov 04 02:07:02.333036 ip-10-0-138-45 kubenswrapper[6073]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:07:02.333036 ip-10-0-138-45 kubenswrapper[6073]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 02:07:02.333036 ip-10-0-138-45 kubenswrapper[6073]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:07:02.333036 ip-10-0-138-45 kubenswrapper[6073]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 02:07:02.335083 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335003 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.335083 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335085 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335088 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335092 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335095 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335098 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335101 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335103 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335110 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335113 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335116 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335119 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335121 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335124 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335127 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335130 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335132 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335135 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335138 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335140 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.335151 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335143 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335146 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335149 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335152 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335155 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335158 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335161 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335163 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335166 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335169 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335171 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335174 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335176 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335179 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335182 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335184 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335186 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335189 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335192 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.335657 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335195 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335198 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335202 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335207 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335211 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335214 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335217 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335220 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335223 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335225 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335228 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335231 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335233 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335236 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335239 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335241 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335244 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335246 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335249 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.336119 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335252 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335254 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335257 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335259 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335262 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335264 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335267 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335269 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335272 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335274 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335277 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335279 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335282 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335285 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335288 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335290 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335293 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335296 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335299 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335301 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.336655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335304 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335307 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335309 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335312 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335314 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335317 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335322 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335325 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335346 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335349 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335352 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335354 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335357 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335359 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335362 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335364 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335367 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335370 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335372 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335375 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.337158 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335377 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335380 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335382 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335386 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335390 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335393 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335397 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.335401 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336133 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336139 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336142 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336146 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336148 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336151 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336154 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336157 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336160 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336162 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336165 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.337666 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336168 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336171 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336173 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336176 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336178 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336181 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336184 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336186 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336189 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336191 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336194 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336196 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336199 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336201 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336204 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336207 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336210 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336213 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336215 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.338129 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336218 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336221 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336231 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336234 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336237 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336239 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336242 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336244 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336247 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336250 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336252 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336255 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336257 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336260 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336262 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336265 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336267 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336270 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336272 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336275 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.338719 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336278 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336280 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336283 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336285 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336288 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336290 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336293 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336295 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336298 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336301 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336304 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336306 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336309 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336311 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336315 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336324 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336343 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336347 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336350 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.339227 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336352 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336355 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336358 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336360 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336363 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336366 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336368 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336370 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336373 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336375 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336378 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336380 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336383 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336385 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336388 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336390 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336393 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336396 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336401 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.339715 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336412 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336415 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336418 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336420 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336424 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336426 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336429 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336432 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336434 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336437 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336441 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336443 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336446 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336449 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336451 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336454 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336458 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336460 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336978 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336983 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.340188 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336986 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336988 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336991 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336993 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336996 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.336999 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337001 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337004 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337007 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337009 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337012 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337014 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337017 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337019 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337022 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337025 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337030 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337033 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337035 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.340723 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337038 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337042 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337046 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337057 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337061 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337064 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337068 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337072 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337076 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337079 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337083 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337086 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337090 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337093 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337097 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337100 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337104 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337107 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337110 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337113 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.341189 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337115 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337118 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337121 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337124 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337126 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337129 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337131 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337134 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337136 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337140 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337143 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337145 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337148 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337150 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337153 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337155 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337164 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337167 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337170 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337173 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.341732 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337175 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337178 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337180 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337182 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337185 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337188 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337190 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337194 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337197 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337199 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337202 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337204 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337207 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337209 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337212 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337214 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337217 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337219 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337222 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.342210 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337224 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337226 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337233 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337236 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337239 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337242 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337247 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337250 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337253 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337255 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337265 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337268 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337270 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337274 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337277 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337280 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337284 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337287 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337290 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.342692 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337292 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337295 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337300 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337302 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337305 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337307 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.337310 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337420 6073 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337435 6073 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337444 6073 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337448 6073 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337453 6073 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337456 6073 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337460 6073 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337464 6073 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337469 6073 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337472 6073 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337476 6073 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337479 6073 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337482 6073 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337485 6073 flags.go:64] FLAG: --cgroup-root="" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337487 6073 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337490 6073 flags.go:64] FLAG: --client-ca-file="" Nov 04 02:07:02.343229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337494 6073 flags.go:64] FLAG: --cloud-provider="external" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337502 6073 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337509 6073 flags.go:64] FLAG: --cluster-domain="" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337512 6073 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337515 6073 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337518 6073 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337521 6073 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337525 6073 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337528 6073 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337531 6073 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337534 6073 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337538 6073 flags.go:64] FLAG: --contention-profiling="false" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337542 6073 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337544 6073 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337548 6073 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337550 6073 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337555 6073 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337557 6073 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337561 6073 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337563 6073 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337566 6073 flags.go:64] FLAG: --enable-server="true" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337569 6073 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337576 6073 flags.go:64] FLAG: --event-burst="100" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337579 6073 flags.go:64] FLAG: --event-qps="50" Nov 04 02:07:02.344236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337583 6073 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337585 6073 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337588 6073 flags.go:64] FLAG: --eviction-hard="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337593 6073 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337595 6073 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337598 6073 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337601 6073 flags.go:64] FLAG: --eviction-soft="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337604 6073 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337607 6073 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337610 6073 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337618 6073 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337621 6073 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337624 6073 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337627 6073 flags.go:64] FLAG: --feature-gates="" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337631 6073 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337635 6073 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337638 6073 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337641 6073 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337644 6073 flags.go:64] FLAG: --healthz-port="10248" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337648 6073 flags.go:64] FLAG: --help="false" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337651 6073 flags.go:64] FLAG: --hostname-override="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337654 6073 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337657 6073 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 02:07:02.345318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337660 6073 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337663 6073 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337666 6073 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337675 6073 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337678 6073 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337681 6073 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337684 6073 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337687 6073 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337690 6073 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337693 6073 flags.go:64] FLAG: --kube-reserved="" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337696 6073 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337698 6073 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337701 6073 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337704 6073 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337707 6073 flags.go:64] FLAG: --lock-file="" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337710 6073 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337712 6073 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337715 6073 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337720 6073 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337723 6073 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337733 6073 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337737 6073 flags.go:64] FLAG: --logging-format="text" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337739 6073 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337743 6073 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 02:07:02.346381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337745 6073 flags.go:64] FLAG: --manifest-url="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337748 6073 flags.go:64] FLAG: --manifest-url-header="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337755 6073 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337759 6073 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337781 6073 flags.go:64] FLAG: --max-pods="110" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337786 6073 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337789 6073 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337793 6073 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337796 6073 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337799 6073 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337802 6073 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337805 6073 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337816 6073 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337819 6073 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337822 6073 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337825 6073 flags.go:64] FLAG: --pod-cidr="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337828 6073 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337831 6073 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337833 6073 flags.go:64] FLAG: --pods-per-core="0" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337836 6073 flags.go:64] FLAG: --port="10250" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337839 6073 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337842 6073 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0f6a3e8edb0efabdd" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337845 6073 flags.go:64] FLAG: --qos-reserved="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337848 6073 flags.go:64] FLAG: --read-only-port="10255" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337851 6073 flags.go:64] FLAG: --register-node="true" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337853 6073 flags.go:64] FLAG: --register-with-taints="" Nov 04 02:07:02.347439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337857 6073 flags.go:64] FLAG: --registry-burst="10" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337860 6073 flags.go:64] FLAG: --registry-qps="5" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337863 6073 flags.go:64] FLAG: --reserved-cpus="" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337871 6073 flags.go:64] FLAG: --reserved-memory="" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337875 6073 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337878 6073 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337881 6073 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337884 6073 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337887 6073 flags.go:64] FLAG: --runonce="false" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337889 6073 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337893 6073 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337896 6073 flags.go:64] FLAG: --seccomp-default="false" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337899 6073 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337902 6073 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337905 6073 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337908 6073 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337911 6073 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337914 6073 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337917 6073 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337919 6073 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337922 6073 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337925 6073 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337928 6073 flags.go:64] FLAG: --system-cgroups="" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337931 6073 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337936 6073 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 02:07:02.348576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337939 6073 flags.go:64] FLAG: --tls-cert-file="" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337941 6073 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337948 6073 flags.go:64] FLAG: --tls-min-version="" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337950 6073 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337953 6073 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337956 6073 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337959 6073 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337962 6073 flags.go:64] FLAG: --v="2" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337966 6073 flags.go:64] FLAG: --version="false" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337970 6073 flags.go:64] FLAG: --vmodule="" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337973 6073 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.337982 6073 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338180 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338187 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338193 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338197 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338201 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338206 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338209 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338212 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338214 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338217 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.349674 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338220 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338222 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338225 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338228 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338230 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338233 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338247 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338249 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338252 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338255 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338257 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338260 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338262 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338265 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338267 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338270 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338273 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338275 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338278 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338280 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.350641 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338283 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338285 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338295 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338298 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338301 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338303 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338306 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338309 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338312 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338314 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338317 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338319 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338322 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338325 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338340 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338345 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338349 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338352 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338361 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.351375 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338365 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338367 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338370 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338373 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338376 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338378 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338381 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338383 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338386 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338388 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338391 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338394 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338396 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338399 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338401 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338404 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338412 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338415 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338417 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338420 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.351947 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338424 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338426 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338429 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338431 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338434 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338437 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338440 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338442 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338445 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338447 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338450 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338454 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338456 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338459 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338461 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338463 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338466 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338468 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338471 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338474 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.352603 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338476 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338479 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338481 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338484 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338487 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338489 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338492 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338494 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338497 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338505 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338509 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338513 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338517 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338520 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338523 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338525 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.353138 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338528 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.338557 6073 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338608 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338613 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338617 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338619 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338623 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338627 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338630 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338633 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338636 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338638 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338641 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338644 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338646 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.353564 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338650 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338667 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338671 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338675 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338678 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338681 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338684 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338686 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338689 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338692 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338694 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338704 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338706 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338709 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338713 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338715 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338718 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338721 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338724 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338726 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.353954 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338729 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338731 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338734 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338737 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338741 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338743 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338746 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338748 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338751 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338753 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338756 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338758 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338760 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338763 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338766 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338768 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338771 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338773 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338776 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338778 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.354709 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338781 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338784 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338788 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338791 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338799 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338802 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338806 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338809 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338811 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338814 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338817 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338819 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338822 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338824 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338827 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338830 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338833 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338836 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338838 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.355198 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338841 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338843 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338846 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338848 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338851 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338854 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338856 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338859 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338861 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338863 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338866 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338869 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338871 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338873 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338876 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338878 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338881 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338884 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338892 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338897 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.355684 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338899 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338902 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338905 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338907 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338910 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338912 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338915 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338917 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338920 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338924 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338926 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338929 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338931 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.338935 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.343894 6073 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.343919 6073 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.343944 6073 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.343952 6073 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344131 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.356181 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344140 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344146 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344151 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344155 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344160 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344165 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344176 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344180 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344185 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344190 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344194 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344199 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344203 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344208 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344212 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344242 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344247 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344252 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344256 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.356710 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344260 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344264 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344268 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344273 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344278 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344283 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344288 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344293 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344298 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344465 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344473 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344478 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344482 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344487 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344491 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344499 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344506 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344512 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344518 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.357156 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344523 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344527 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344531 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344536 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344542 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344547 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344551 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344555 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344561 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344566 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344571 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344576 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344580 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344584 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344588 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344593 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344597 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344601 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344606 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.357647 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344610 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344614 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344619 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344623 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344627 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344631 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344636 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344640 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344645 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344649 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344653 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344657 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344662 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344666 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344670 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344675 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344679 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344683 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344687 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344692 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.358167 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344696 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344701 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344706 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344710 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344714 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344718 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344723 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344727 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344731 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344735 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344739 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344744 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344748 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344752 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344756 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344760 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344764 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344769 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344773 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344777 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.358699 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344781 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344785 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344789 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344794 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344798 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344803 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344807 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.344811 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.344856 6073 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345086 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345095 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345100 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345105 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345109 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345124 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.359182 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345129 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345133 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345137 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345144 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345152 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345157 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345162 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345166 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345170 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345174 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345178 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345183 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345188 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345192 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345196 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345200 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345204 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345208 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345213 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.359842 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345217 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345221 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345225 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345229 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345233 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345237 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345242 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345246 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345250 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345254 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345259 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345263 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345268 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345274 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345279 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345283 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345288 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345292 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345296 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345300 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.360521 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345304 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345308 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345312 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345316 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345320 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345324 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345353 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345358 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345362 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345366 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345370 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345375 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345379 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345383 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345387 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345392 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345396 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345400 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345405 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.361269 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345409 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345413 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345417 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345422 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345426 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345430 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345435 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345440 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345445 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345449 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345453 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345457 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345462 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345466 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345470 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345477 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345483 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345488 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345493 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.361863 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345497 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345501 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345505 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345510 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345514 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345519 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345523 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345527 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345531 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345535 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345539 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345544 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345548 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345552 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345556 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345560 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345564 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345569 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345573 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345577 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.362387 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345582 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345587 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345591 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.345633 6073 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345686 6073 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345695 6073 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345702 6073 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345707 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345711 6073 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345716 6073 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345725 6073 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345729 6073 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345734 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345738 6073 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345743 6073 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 02:07:02.362876 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345747 6073 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345752 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345756 6073 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345760 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345765 6073 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345769 6073 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345774 6073 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345778 6073 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345782 6073 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345786 6073 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345790 6073 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345794 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345799 6073 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345803 6073 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345807 6073 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345812 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345817 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345821 6073 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345826 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345832 6073 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 02:07:02.363512 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345836 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345841 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345845 6073 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345849 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345853 6073 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345858 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345862 6073 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345868 6073 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345872 6073 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345876 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345880 6073 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345885 6073 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345889 6073 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345893 6073 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345897 6073 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345901 6073 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345906 6073 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345910 6073 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345915 6073 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 02:07:02.364263 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345919 6073 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345923 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345927 6073 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345931 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345935 6073 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345939 6073 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345944 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345949 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345954 6073 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345958 6073 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345963 6073 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345967 6073 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345971 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345976 6073 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345980 6073 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345984 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345989 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345993 6073 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.345997 6073 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 02:07:02.364760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346002 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346008 6073 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346014 6073 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346020 6073 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346024 6073 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346029 6073 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346033 6073 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346038 6073 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346042 6073 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346046 6073 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346051 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346055 6073 feature_gate.go:386] unrecognized feature gate: Example Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346059 6073 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346063 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346067 6073 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346071 6073 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346075 6073 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346080 6073 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346084 6073 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346088 6073 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 02:07:02.365221 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346092 6073 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346096 6073 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346100 6073 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346105 6073 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346111 6073 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346115 6073 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346119 6073 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346124 6073 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346129 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346133 6073 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346137 6073 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346141 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346146 6073 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346152 6073 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346157 6073 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346162 6073 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:02.346166 6073 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.346283 6073 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 02:07:02.365754 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.347852 6073 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.347934 6073 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.348207 6073 server.go:1014] "Starting client certificate rotation" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.348352 6073 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.348411 6073 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:00:51 +0000 UTC" deadline="2028-04-19 20:40:08.79358806 +0000 UTC" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.348444 6073 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12786h33m6.445147231s" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.348954 6073 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.350734 6073 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.351988 6073 log.go:25] "Validated CRI v1 runtime API" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.354048 6073 log.go:25] "Validated CRI v1 image API" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.354792 6073 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 02:07:02.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.357303 6073 fs.go:135] Filesystem UUIDs: map[2bf98269-6ffb-4ebb-bfd8-22370da6d87e:/dev/nvme0n1p4 400bd14d-ef8a-4f68-8b76-24f7a16f5ad4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Nov 04 02:07:02.366523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.357322 6073 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4/userdata/shm major:0 minor:544 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25/userdata/shm major:0 minor:257 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d/userdata/shm major:0 minor:540 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae/userdata/shm major:0 minor:248 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a/userdata/shm major:0 minor:370 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c/userdata/shm major:0 minor:353 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0/userdata/shm major:0 minor:301 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1/userdata/shm major:0 minor:496 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264/userdata/shm major:0 minor:315 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9/userdata/shm major:0 minor:455 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1/userdata/shm major:0 minor:407 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264/userdata/shm major:0 minor:542 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5/userdata/shm major:0 minor:359 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~projected/kube-api-access-mnpfm:{mountpoint:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~projected/kube-api-access-mnpfm major:0 minor:495 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:490 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:494 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30429b8d-d02a-4d8c-a500-a3bbf17e7a61/volumes/kubernetes.io~projected/kube-api-access-jnczh:{mountpoint:/var/lib/kubelet/pods/30429b8d-d02a-4d8c-a500-a3bbf17e7a61/volumes/kubernetes.io~projected/kube-api-access-jnczh major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~projected/kube-api-access-st24x:{mountpoint:/var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~projected/kube-api-access-st24x major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~projected/kube-api-access-tjq8c:{mountpoint:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~projected/kube-api-access-tjq8c major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~projected/kube-api-access-2ljpv:{mountpoint:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~projected/kube-api-access-2ljpv major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~projected/kube-api-access-d78kv:{mountpoint:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~projected/kube-api-access-d78kv major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:365 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~projected/kube-api-access-kjl6g:{mountpoint:/var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~projected/kube-api-access-kjl6g major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72d28c97-8c3a-4ccf-8156-c496da17cbea/volumes/kubernetes.io~projected/kube-api-access-fxmm2:{mountpoint:/var/lib/kubelet/pods/72d28c97-8c3a-4ccf-8156-c496da17cbea/volumes/kubernetes.io~projected/kube-api-access-fxmm2 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/803bdfd3-1098-4104-abc0-91f2b5ec5192/volumes/kubernetes.io~projected/kube-api-access-mmnhc:{mountpoint:/var/lib/kubelet/pods/803bdfd3-1098-4104-abc0-91f2b5ec5192/volumes/kubernetes.io~projected/kube-api-access-mmnhc major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~projected/kube-api-access-n95zw:{mountpoint:/var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~projected/kube-api-access-n95zw major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/config-reloader/2 major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/prometheus/15 major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~empty-dir/config-out major:0 minor:443 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/kube-api-access-lljqf:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/kube-api-access-lljqf major:0 minor:454 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/tls-assets major:0 minor:442 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/config major:0 minor:435 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:440 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:438 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:439 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:433 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:441 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:437 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/web-config major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~projected/kube-api-access-b24rv:{mountpoint:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~projected/kube-api-access-b24rv major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~projected/kube-api-access-fxlk6:{mountpoint:/var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~projected/kube-api-access-fxlk6 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~secret/metrics-certs major:0 minor:538 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~projected/kube-api-access-c7g5l:{mountpoint:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~projected/kube-api-access-c7g5l major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~projected/kube-api-access-hcm2z:{mountpoint:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~projected/kube-api-access-hcm2z major:0 minor:403 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:396 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a50e239d-68ef-4cde-88ab-ee15083433d2/volumes/kubernetes.io~projected/kube-api-access-bnzrd:{mountpoint:/var/lib/kubelet/pods/a50e239d-68ef-4cde-88ab-ee15083433d2/volumes/kubernetes.io~projected/kube-api-access-bnzrd major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb61a6a8-f7ee-4140-8dff-1b292b2173b8/volumes/kubernetes.io~projected/kube-api-access-ddzdg:{mountpoint:/var/lib/kubelet/pods/bb61a6a8-f7ee-4140-8dff-1b292b2173b8/volumes/kubernetes.io~projected/kube-api-access-ddzdg major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:529 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c89b664a-dd5d-4631-bbae-d197c38825e2/volumes/kubernetes.io~projected/kube-api-access-l4jzz:{mountpoint:/var/lib/kubelet/pods/c89b664a-dd5d-4631-bbae-d197c38825e2/volumes/kubernetes.io~projected/kube-api-access-l4jzz major:0 minor:539 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~projected/kube-api-access-2plqx:{mountpoint:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~projected/kube-api-access-2plqx major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~projected/kube-api-access-kwhts:{mountpoint:/var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~projected/kube-api-access-kwhts major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~projected/kube-api-access-78rp4:{mountpoint:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~projected/kube-api-access-78rp4 major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e114adad-34e0-4007-9a4c-011c60df73c6/volumes/kubernetes.io~projected/kube-api-access-nhvjf:{mountpoint:/var/lib/kubelet/pods/e114adad-34e0-4007-9a4c-011c60df73c6/volumes/kubernetes.io~projected/kube-api-access-nhvjf major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/alertmanager/9 major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/config-reloader/6 major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~empty-dir/config-out major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/kube-api-access-9j7gz:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/kube-api-access-9j7gz major:0 minor:311 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/tls-assets major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/config-volume major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/web-config major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~projected/kube-api-access-vzdhz:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~projected/kube-api-access-vzdhz major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~projected/kube-api-access-hpmfm:{mountpoint:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~projected/kube-api-access-hpmfm major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc9507ca-eeeb-4ed3-ab31-019380293e75/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/fc9507ca-eeeb-4ed3-ab31-019380293e75/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/6ea557409095423387943bbbaa299fc7dbc9685d70d7b474e1f08db072796131/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/77dee2f51556eeadea6b0d35553025778187b6931339fcf2259b6d0d23c70034/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/47f7b6ccd7dbf557220e2a17306058daf39ed3a9e3fcc0d6dca112a5a5393ca8/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/b470a4dac8b8f99d1bf16401f647c7867704967fbc3993e85f682268e7026cc0/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/081a2c8024e71591b8d12669b77a2dd720a61d8cee7a1958209a8386bdaf0fc6/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/6b897746a7c9353e936880f57fb9912232b25853a07aaa99ec12328a3af4221a/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-122:{mountpoint:/var/lib/containers/storage/overlay/243a1ac800325f6cd015f24d70ff994cfae66c68af764d216d216dd7bf700f68/merged major:0 minor:122 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/5c66c86ea6bcf6867989eff86052b4f81da3aa682849ccdd50bc28ba1ecd6160/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/b0a9dd91abd4fd1ac517ce49eac631e779775039b716a14f46fd1d3839202c66/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/bbad5e0f4f9e4fdfa586eda1bf35f025ba9f5cec80d46f828c4958fff55ffe59/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/df065468d01c8f7a9f7923e3431a4b171e6f3d2ba333f605c272e1de33bca05f/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/b8fbabb99472da06a34aa47f497ddfaec8bbefec1f96df8145d96dc712937a2e/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/a060d3e5d74e1532fdddbc07bc43f0ebc75d623a9120829caa09b474015e92b2/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/3515eb0f2c6eb6a1e32ae08265fe1afc009b38f664da08ca3930bb6722df7e28/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/abda52eb45fed6058e11025e47bad21cb15b7cc51af6c1d460149afc69665fc5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/9f43921484fe2e2799fb1d80d446a1538df7004dbef32ecd189805ed52904040/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/4f8c3ae66ce2b62cf0d8665b0805c3870fcec0924a00cfcf3cdeb2e805989830/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/a0b79be58d0c4b74d3f8d0b108df218db099d719831d27e529a7d64f39364aac/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b2fc7952b895b616e22c950c6d14c2447009fda494ed000ee58a907a83d2d1cd/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/65aaa81135bc295423ed71469f770eb41d678bd2f927d8126072a0c0ecaf9ae5/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/ee5de04d117639eabb83c0fc840b488fd1e3a39556cf7abedcc90048ae6d2aa1/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/4db5851753ef6f93f506672f8b5ed964127ac80c25b302eb14db98848f32c45f/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/b728180a5bacef800ac77c47c1d6f2d390e280bf912bd1b75d8d2febee94a209/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/7080f8c575c8cc1ac70a4923dec12b5d285ee050b1addaecf83ce53c83ff5be2/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/5b10f6cc4e02d92869d8c1b423fdcdcaa10d3d5da7c352f7528cfddcad7aaea9/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/0d46c37e7fa3f63306b1b67693ae9a4d20cd5119baa2aa12f907c5336e2e0ee5/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/0280dd66db5e2aca7dc4d22e58dd25e972d1b0299586b1d21daa97a20d255b52/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/2d5354a6c870793e827c2ca3a1c8789df104498609c14bc7d51b7cad08ac48c1/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/ea0ae8ceb1a504fca1be95f44338ac7ba381a5d48fd12ef9cd2fcbd424035679/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c51c6763a1c1c74b449f1c34e435a96243c9d95ad28d2355994c2a962b804e41/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/232492dab5a92c88523037570b2424a780f3951f40907a5c252639a0f994ffc3/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/525c08cb7e8f54c6c664e9dd90ffe0ddf0191b8d464075e702b7fe4d6a20320e/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/8e2c88d228d73090b9d52c279577d23274901657925f65ae12327380c91b25ba/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/d111891f49cffe8ab43610d75d1946ce5cf9a4425191c23940c35d9b451023a9/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/0f5004be16b631b9c647b6ce06a7c59719f9eb10a552e08c35c593fa293cc7c2/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/551f28b1aca55a3efe78a8eede798f71f2ab46faf3e30a44f1bb539ef6bbdc99/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/1d7fe2a629bbd11fd02fa05d5f1219527cd712280727ba57152416c62ab787fd/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/ae509ec655f402bd92f3a87066187ccca424fc5fd6d86f5e3c6878e82cf3c2ff/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/f132b95ac81b8f68426a16ee48629feb2e7146596f237813b9ab0870d72ce037/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/0b284da30d846e2f9f2d94c6301184352f3682fe7897cdfaa83772766c6516d6/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/822f1ed047f275f0d0f7a6436a198a181de7238a720bbd4e154811042972bbe9/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/90efdb4100e30b264841c6de235a37532823f2c85e4f0f45a94499e92ffca93a/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/d487a6019afd1f029935b7c9204b998bdc0aee8c40096f4fa01d353ae7ee219a/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-341:{mountpoint:/var/lib/containers/storage/overlay/54ac499808857595cf3baf363aa7fa6e0a2a03ed73594d3e06e44504ba7bc361/merged major:0 minor:341 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/a05028c7035615d34701cba9bc137aa9342f987d3736ade6ebd737edf7f24fc7/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/67e8ddfdb0bcfb9e4cf9038fb284d194226b3a9bb9e8210bb6393b5ff0f6cd23/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/eb36bb54d5ff1becc943c469f569da4294c21bd623ea4aecfcb834c70ad7a1e7/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-372:{mountpoint:/var/lib/containers/storage/overlay/375945872a7ee4749ff6f3a196b1c9c1b48701428cffc430a997011c1724ce8d/merged major:0 minor:372 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/a25e992e3825d54ae7e4fe1c6cd572d023fe0c165b91d2b8963b303ae6651d65/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/ed23bac82487af88ef1e2e4d50f35c5417d2255d6723c0364a1b74349109fed2/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/983edc78f16e9c28d59da25a1cedc61d01a0a0d960585de69952f9fe187159fa/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/7e3f885bbbf78508a63f0540482d83cc2ebc2d30e094e4bef8b58884cdc283c6/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/c8cc8e6082979abebb7cf33f091af0db5517f8b66a77fb08b6a56e6ae852368c/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/e72ec1141858eb7b446ff5e1c0ed12b0a18ef3a9db81835e11a6ec21a1dfa9fa/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/7e1a20bcf7ac022f456f0487e2429a6913147b33049c25a6beac9d4a21a50f57/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/399132f61a1cf5f7ef1c15fa74dcf8eabda8e2bdb793e9c826daa05d9b6753ec/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/67d9ac181a4f24cf9687ff0b865de8d19ceced1f616a87fa477ed067b3823965/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/411c10f8284d8dd114b023962489329cf436074d66bfbd20902482b244a8dc36/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/4079789d66faef5796957fe06cb71a9cfcd2be43663e5546247cc483b101d919/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/da8ecbe7d2b5ddcebc9daaf305c5161d645c74b8322ac9806957f0a7fb9d722a/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-449:{mountpoint:/var/lib/containers/storage/overlay/b73b092012316f6455d71bf15bb585eb3d64c5a56456e625b43f3724c7fb3ad1/merged major:0 minor:449 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/9d1ed7998675f683a820ffc5b744ad0ae1a8a88fc1e3320b5219f7efb3b08750/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/84293789c63ee5ad110961d9c11dafd42dcf8d8f9b1821bb46eaeff330c66ae6/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-468:{mountpoint:/var/lib/containers/storage/overlay/20f2520ecd65209af0e2d9afe0cfdf9c080fe2e1d789e4867293eb0a569bcb6d/merged major:0 minor:468 fsType:overlay blockSize:0} overlay_0-469:{mountpoint:/var/lib/containers/storage/overlay/ed385cc9219ece9267c67f15fe21c85d12f721cd2a9f5c52327666ea3015cf63/merged major:0 minor:469 fsType:overlay blockSize:0} overlay_0-475:{mountpoint:/var/lib/containers/storage/overlay/3d0f3dd3555ec0c83e9fcba968cb259c832412f2189e19057c3b52955484d1e4/merged major:0 minor:475 fsType:overlay blockSize:0} overlay_0-477:{mountpoint:/var/lib/containers/storage/overlay/3086810d2f6358c99f9028a9acaac97504bf6ef40b2af3d2b03f7631e6df27af/merged major:0 minor:477 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/a7d22619dbda87b38c92573f213f326bfa6da36d497e346a6bc9e02040e56b43/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-498:{mountpoint:/var/lib/containers/storage/overlay/0b6d1dc63da36bfe378345199d11747b66eb98d00ac112709d310962cd81ef27/merged major:0 minor:498 fsType:overlay blockSize:0} overlay_0-500:{mountpoint:/var/lib/containers/storage/overlay/396770aab7b578c2cfda1ca7a71ebd6aedde6652eb01fa48eb1bc4d535aa43a8/merged major:0 minor:500 fsType:overlay blockSize:0} overlay_0-506:{mountpoint:/var/lib/containers/storage/overlay/41dd26be3098a555100da3e3cccb1653ee45fefdc1b8674a5a998789c93f1bd0/merged major:0 minor:506 fsType:overlay blockSize:0} overlay_0-511:{mountpoint:/var/lib/containers/storage/overlay/13b1d09ed675fc2db80afaaa19148e129abbb3e00528ab49692695ecc856f703/merged major:0 minor:511 fsType:overlay blockSize:0} overlay_0-516:{mountpoint:/var/lib/containers/storage/overlay/b622c3d4a556e12c457e30a2a03cf61c82b78fb506ec15399466229cf8e01b64/merged major:0 minor:516 fsType:overlay blockSize:0} overlay_0-521:{mountpoint:/var/lib/containers/storage/overlay/0f346a1e5bf95d5b28cb371267bd96d54429ca7b7f554193c2293891c6ceff9d/merged major:0 minor:521 fsType:overlay blockSize:0} overlay_0-524:{mountpoint:/var/lib/containers/storage/overlay/e2ec46afa274b5953ad131eb55258096ca014949ce8c32e4324b847870fd8736/merged major:0 minor:524 fsType:overlay blockSize:0} overlay_0-533:{mountpoint:/var/lib/containers/storage/overlay/4c5160a8d5ab9fbc7b2a3b81a34b402cafa0b9f16eb503804486d297debef681/merged major:0 minor:533 fsType:overlay blockSize:0} overlay_0-546:{mountpoint:/var/lib/containers/storage/overlay/83f4b061787591259404d2a66e565368997736da1765ffef7babaca7f9386cb4/merged major:0 minor:546 fsType:overlay blockSize:0} overlay_0-548:{mountpoint:/var/lib/containers/storage/overlay/c50529b039c4cd894c56b0605f41af9fcab392f70f0f3a05efacdbe9c7c01c8c/merged major:0 minor:548 fsType:overlay blockSize:0} overlay_0-550:{mountpoint:/var/lib/containers/storage/overlay/547e49c5268400d77335a8a0e1bf6077fb6235cddc69da6643dac9339f70973e/merged major:0 minor:550 fsType:overlay blockSize:0} overlay_0-552:{mountpoint:/var/lib/containers/storage/overlay/c26970d40b85262a6fb6a046702a2e3fa64d718766bfc42c70f0f4a93d708608/merged major:0 minor:552 fsType:overlay blockSize:0} overlay_0-554:{mountpoint:/var/lib/containers/storage/overlay/867a051de73f17901bc938556d241018262bfbe91a88c3796e18fe1803a6e535/merged major:0 minor:554 fsType:overlay blockSize:0} overlay_0-560:{mountpoint:/var/lib/containers/storage/overlay/174eca8b84e21434d87bc6c0bb27fb17456f964423cbf6e563aa7368def3673d/merged major:0 minor:560 fsType:overlay blockSize:0} overlay_0-565:{mountpoint:/var/lib/containers/storage/overlay/c15277b075ec30975f6e24c069d3d36ab57236d1c211d1f1c47772eab1706087/merged major:0 minor:565 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/3e2e53805c2f8fa98bc762d66a11b5b8588a7ef050b7dea0335cb44c90d79d44/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/5a6598054489874fe1c7d5d0cbe91115043d846a0e37568081e1bbb347df3b26/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/5f5b6235f90cbf55805b545a56b5f994fc37e9fbfedc37a6878503ab89ee231c/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/61ff4791d0055569750c10d4ee05b046f4d559e558d3a82536b8000df24f7695/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{moun Nov 04 02:07:02.366825 ip-10-0-138-45 kubenswrapper[6073]: tpoint:/var/lib/containers/storage/overlay/50741387fb3130f1689b9b1f88fc8fb138aa6be858711f84b2d4e0c9084fec90/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/249d3ed2028902750759442e488fb66f00178504fc219ced61f6fb604da8864e/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/24b0582215c654ba00f5d56b7bc2baf9c097e922747679110f91a19df50bced8/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/ef517d257d1faffa4fbd0c9a792a9dc9177243ae37639f10bdadb2303818fd2a/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/2a8b7bf8c047944369043f6e3179eb7102bbf70ae01b1af801420706da1a2e07/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/ae602bfa36c7415d46b46f6dadd2cf9102f5291ade3e07632b63af2de6f91ac5/merged major:0 minor:98 fsType:overlay blockSize:0}] Nov 04 02:07:02.368087 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.367564 6073 manager.go:217] Machine: {Timestamp:2026-11-04 02:07:02.367074886 +0000 UTC m=+0.062765077 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098987 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec244f76d8a432ad5fc92bf111e8a982 SystemUUID:ec244f76-d8a4-32ad-5fc9-2bf111e8a982 BootID:034ac16a-29db-4309-b330-0584587c2eae Filesystems:[{Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:304 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~projected/kube-api-access-n95zw DeviceMajor:0 DeviceMinor:352 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-524 DeviceMajor:0 DeviceMinor:524 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb7fbb348702aa296361b68067d6efe733c30187c26c35f04833349cc60f6264/userdata/shm DeviceMajor:0 DeviceMinor:542 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:365 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/30429b8d-d02a-4d8c-a500-a3bbf17e7a61/volumes/kubernetes.io~projected/kube-api-access-jnczh DeviceMajor:0 DeviceMinor:54 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/180ee2cf8528a33113eae5b5c83fd0f01fb767fb0ebeff93522086bb4aa4de14/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:239 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:442 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:434 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-511 DeviceMajor:0 DeviceMinor:511 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-516 DeviceMajor:0 DeviceMinor:516 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~projected/kube-api-access-kwhts DeviceMajor:0 DeviceMinor:203 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:212 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-477 DeviceMajor:0 DeviceMinor:477 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb61a6a8-f7ee-4140-8dff-1b292b2173b8/volumes/kubernetes.io~projected/kube-api-access-ddzdg DeviceMajor:0 DeviceMinor:53 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-122 DeviceMajor:0 DeviceMinor:122 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:240 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~projected/kube-api-access-d78kv DeviceMajor:0 DeviceMinor:369 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-372 DeviceMajor:0 DeviceMinor:372 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:286 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:286 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-475 DeviceMajor:0 DeviceMinor:475 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/fc9507ca-eeeb-4ed3-ab31-019380293e75/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/85a536552962023b2594f7f08674328d1ee584ca15648c7e6a962e597a1aa439/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/97e54f8004dc9911db428f47ff9e6ed454ab21751f30f8e186c451d97b64fa92/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/kube-api-access-9j7gz DeviceMajor:0 DeviceMinor:311 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:356 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:434 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:439 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1a2f0b2b4efbe813875ea5303976a745c8b323d763e0d10576975402a23bbe25/userdata/shm DeviceMajor:0 DeviceMinor:257 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~projected/kube-api-access-vzdhz DeviceMajor:0 DeviceMinor:313 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-468 DeviceMajor:0 DeviceMinor:468 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:434 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e114adad-34e0-4007-9a4c-011c60df73c6/volumes/kubernetes.io~projected/kube-api-access-nhvjf DeviceMajor:0 DeviceMinor:55 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:295 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~projected/kube-api-access-hcm2z DeviceMajor:0 DeviceMinor:403 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9/userdata/shm DeviceMajor:0 DeviceMinor:455 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/c89b664a-dd5d-4631-bbae-d197c38825e2/volumes/kubernetes.io~projected/kube-api-access-l4jzz DeviceMajor:0 DeviceMinor:539 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-554 DeviceMajor:0 DeviceMinor:554 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~projected/kube-api-access-fxlk6 DeviceMajor:0 DeviceMinor:51 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~projected/kube-api-access-hpmfm DeviceMajor:0 DeviceMinor:213 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1/userdata/shm DeviceMajor:0 DeviceMinor:407 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:436 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a50e239d-68ef-4cde-88ab-ee15083433d2/volumes/kubernetes.io~projected/kube-api-access-bnzrd DeviceMajor:0 DeviceMinor:56 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:293 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:433 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-546 DeviceMajor:0 DeviceMinor:546 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-552 DeviceMajor:0 DeviceMinor:552 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:232 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/29afba3097387722d7a79b69580b406a9c4bb8c76bde38410e913d47f1c5a36d/userdata/shm DeviceMajor:0 DeviceMinor:540 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-550 DeviceMajor:0 DeviceMinor:550 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-565 DeviceMajor:0 DeviceMinor:565 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/72d28c97-8c3a-4ccf-8156-c496da17cbea/volumes/kubernetes.io~projected/kube-api-access-fxmm2 DeviceMajor:0 DeviceMinor:58 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:312 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:357 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~projected/kube-api-access-tjq8c DeviceMajor:0 DeviceMinor:358 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:440 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~projected/kube-api-access-2ljpv DeviceMajor:0 DeviceMinor:57 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:308 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:310 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-341 DeviceMajor:0 DeviceMinor:341 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5125be9a6447db9f7f99ae7327c9eb86cf6e413fa6d917c3f71c3429d089316a/userdata/shm DeviceMajor:0 DeviceMinor:370 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3889db9a-bb9d-422e-a8a6-ffda0e64d867/volumes/kubernetes.io~projected/kube-api-access-st24x DeviceMajor:0 DeviceMinor:202 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~projected/kube-api-access-c7g5l DeviceMajor:0 DeviceMinor:247 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:309 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:297 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f99117f9a60e3efba9d3fdede0b521e2f41da5b3a5cea0324ce5ff30b50f92e5/userdata/shm DeviceMajor:0 DeviceMinor:359 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:366 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:494 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:298 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a5a79f3c5be2131d67bff41ad306fe7447ae5b23049753c008a7db2137bb3621/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:241 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:396 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:435 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/050df69d0bda68e5d39bd064ddde2731deded3013706fb11133c306c95e25cd2/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/269c982072fec449a8058c5d0304c681da195b547108a5be316a1cae75e05a75/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:438 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:443 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/110fadfd364a3502b4cf0a5b796dedb27fa7456b4a1cc5c8f09ec6906135b0c4/userdata/shm DeviceMajor:0 DeviceMinor:544 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/85ce754345cdb55a343c7fa1fd973f5f3da49dd3210fdb8fea844428a8d5931f/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-498 DeviceMajor:0 DeviceMinor:498 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/66f97f7efa25141626f55e2bf8d05defeea39e702dd23840c8869979612448bc/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2e90242-f97c-45f4-abd7-4493c8d9cc42/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:303 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:392 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-560 DeviceMajor:0 DeviceMinor:560 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/49c62d7ad2ddb6b0d3c78a9aab3f21c8226d38c060d5d3765e618d881b8d0ff9/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~projected/kube-api-access-2plqx DeviceMajor:0 DeviceMinor:237 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:490 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-500 DeviceMajor:0 DeviceMinor:500 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/66fdb210-e77b-4433-a0c3-0ac8262cdcb9/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dae263618cf95b9761d30e3c8764041355ed47d1ce157a92b3c8380f901ab62c/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100/volumes/kubernetes.io~projected/kube-api-access-b24rv DeviceMajor:0 DeviceMinor:52 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/803bdfd3-1098-4104-abc0-91f2b5ec5192/volumes/kubernetes.io~projected/kube-api-access-mmnhc DeviceMajor:0 DeviceMinor:59 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:294 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/825cfc8b-20b7-481c-beaa-46a414be370a/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:348 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/caf92aa8-a824-4089-8fa8-9005b529346f/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:236 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:286 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/72d65a984bd5cc79a599161e0c93bf65f7b748cc60beb465560e4069b55f6b6c/userdata/shm DeviceMajor:0 DeviceMinor:353 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:368 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1/userdata/shm DeviceMajor:0 DeviceMinor:496 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9082f258-87ca-4149-9e1e-69e87f0b2cb6/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:538 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-548 DeviceMajor:0 DeviceMinor:548 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3c79071f6548bb2a8a224d70092ee060fd4e0da001110ad582e4c8543225e80/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/40a78f67-2bc4-4d66-b09a-e93063641f64/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:355 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:293 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6cc1b1d9206091bf77c36a59f5054a03c360bc039c16b811a5fc83e4c081ec49/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:291 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9fb066cc449f44e5f9d897cca9f5535d0c3d9f0553d2b698cb81782d3fc38264/userdata/shm DeviceMajor:0 DeviceMinor:315 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:434 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0129d90-8271-46a8-b92b-e179632baf01/volumes/kubernetes.io~projected/kube-api-access-78rp4 DeviceMajor:0 DeviceMinor:244 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:292 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0/userdata/shm DeviceMajor:0 DeviceMinor:301 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes/kubernetes.io~projected/kube-api-access-mnpfm DeviceMajor:0 DeviceMinor:495 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:296 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:437 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~projected/kube-api-access-lljqf DeviceMajor:0 DeviceMinor:454 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-469 DeviceMajor:0 DeviceMinor:469 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70b193ea-8136-4bb9-b98b-a3930e51f99c/volumes/kubernetes.io~projected/kube-api-access-kjl6g DeviceMajor:0 DeviceMinor:204 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:286 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:441 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-449 DeviceMajor:0 DeviceMinor:449 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-506 DeviceMajor:0 DeviceMinor:506 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:211 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9d57b05f-6e23-42fb-b3ff-25479d15b6ae/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:238 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2dea274ad175c927bab359ba63385e37405825f15673c240d2a3dcab9311d6ae/userdata/shm DeviceMajor:0 DeviceMinor:248 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/7014e7c0-9d7d-4306-8e68-98437feeaf4e/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:367 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-521 DeviceMajor:0 DeviceMinor:521 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-533 DeviceMajor:0 DeviceMinor:533 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:529 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:110fadfd364a350 MacAddress:5e:30:0b:c5:79:94 Speed:10000 Mtu:8901} {Name:1a2f0b2b4efbe81 MacAddress:4e:dd:bd:70:04:2c Speed:10000 Mtu:8901} {Name:1d7ada4fcd5a200 MacAddress:c2:7b:97:b9:6c:0e Speed:10000 Mtu:8901} {Name:29afba309738772 MacAddress:de:fb:d0:60:f7:e1 Speed:10000 Mtu:8901} {Name:5125be9a6447db9 MacAddress:46:52:83:09:93:65 Speed:10000 Mtu:8901} {Name:72d65a984bd5cc7 MacAddress:f6:ad:34:82:a0:dc Speed:10000 Mtu:8901} {Name:752921911cbca91 MacAddress:9a:74:d6:c7:b3:af Speed:10000 Mtu:8901} {Name:85a536552962023 MacAddress:e2:ca:49:cd:34:3b Speed:10000 Mtu:8901} {Name:85ce754345cdb55 MacAddress:3e:90:14:f0:eb:3b Speed:10000 Mtu:8901} {Name:8f867fe6cedd34c MacAddress:aa:ec:6f:b0:30:cd Speed:10000 Mtu:8901} {Name:97e54f8004dc991 MacAddress:ae:51:c5:ac:56:47 Speed:10000 Mtu:8901} {Name:9fb066cc449f44e MacAddress:d6:32:83:25:9b:ea Speed:10000 Mtu:8901} {Name:b76f0b1a5087d14 MacAddress:56:41:f0:59:9a:d3 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:46:6a:27:f6:1f Speed:0 Mtu:9001} {Name:br-int MacAddress:be:24:0f:8f:32:ff Speed:0 Mtu:8901} {Name:dae263618cf95b9 MacAddress:5e:4d:79:cd:37:25 Speed:10000 Mtu:8901} {Name:ea8cd51a79791b8 MacAddress:9e:97:53:10:c9:f7 Speed:10000 Mtu:8901} {Name:eb7fbb348702aa2 MacAddress:62:52:5b:5e:19:91 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:46:6a:27:f6:1f Speed:0 Mtu:9001} {Name:f99117f9a60e3ef MacAddress:76:39:e3:4c:a7:b8 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:7e:ac:97:48:44:30 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2e:55:31:ba:e8:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 02:07:02.368374 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368093 6073 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 02:07:02.368374 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368140 6073 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 02:07:02.368505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368476 6073 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 02:07:02.368633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368506 6073 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-45.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 02:07:02.368674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368636 6073 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 02:07:02.368674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368644 6073 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 02:07:02.368674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368654 6073 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 02:07:02.368747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368674 6073 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 02:07:02.368781 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368764 6073 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 02:07:02.368932 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.368923 6073 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 02:07:02.369048 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369040 6073 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 02:07:02.369080 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369077 6073 kubelet.go:493] "Attempting to sync node with API server" Nov 04 02:07:02.369107 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369090 6073 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 02:07:02.369107 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369100 6073 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 02:07:02.369159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369109 6073 kubelet.go:405] "Adding apiserver pod source" Nov 04 02:07:02.369159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369117 6073 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 02:07:02.369822 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369808 6073 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 02:07:02.369822 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.369821 6073 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 02:07:02.370377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.370356 6073 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 02:07:02.373188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373174 6073 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 02:07:02.373188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373188 6073 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 02:07:02.373304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373295 6073 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 02:07:02.373558 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373546 6073 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 02:07:02.373592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373577 6073 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 02:07:02.373687 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373677 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 02:07:02.373717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373693 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 02:07:02.373717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373699 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 02:07:02.373717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373704 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 02:07:02.373717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373712 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373721 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373732 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373739 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373747 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373753 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373766 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373775 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373801 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 02:07:02.373858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.373810 6073 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 02:07:02.377341 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.377312 6073 server.go:1263] "Started kubelet" Nov 04 02:07:02.377419 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.377391 6073 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 02:07:02.377904 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.377840 6073 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 02:07:02.377893 ip-10-0-138-45 systemd[1]: Started Kubernetes Kubelet. Nov 04 02:07:02.378159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.377954 6073 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 02:07:02.378923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.378673 6073 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 02:07:02.382488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.382451 6073 server.go:317] "Adding debug handlers to kubelet server" Nov 04 02:07:02.387194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387165 6073 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 02:07:02.387305 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387197 6073 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387390 6073 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:52 +0000 UTC" deadline="2028-05-05 06:20:29.364584819 +0000 UTC" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387419 6073 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13156h13m26.977168794s" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387438 6073 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387453 6073 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.387455 6073 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:07:02.388354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.387766 6073 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 02:07:02.389061 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389042 6073 factory.go:55] Registering systemd factory Nov 04 02:07:02.389134 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389063 6073 factory.go:223] Registration of the systemd container factory successfully Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389626 6073 factory.go:153] Registering CRI-O factory Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389645 6073 factory.go:223] Registration of the crio container factory successfully Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389705 6073 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389728 6073 factory.go:103] Registering Raw factory Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.389747 6073 manager.go:1199] Started watching for new ooms in manager Nov 04 02:07:02.390572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.390116 6073 manager.go:319] Starting recovery of all containers Nov 04 02:07:02.391223 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391177 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config" seLinuxMountContext="" Nov 04 02:07:02.391223 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391213 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391229 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72d28c97-8c3a-4ccf-8156-c496da17cbea" volumeName="kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391241 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391253 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391264 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391277 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391289 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50e239d-68ef-4cde-88ab-ee15083433d2" volumeName="kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391303 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391315 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391347 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ced39f9f-ae73-431b-aeb4-bb0b2ae2b218" volumeName="kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391359 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume" seLinuxMountContext="" Nov 04 02:07:02.391368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391370 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391383 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391666 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391681 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391692 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391714 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391726 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391740 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391751 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391763 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391775 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caf92aa8-a824-4089-8fa8-9005b529346f" volumeName="kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391792 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391805 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b193ea-8136-4bb9-b98b-a3930e51f99c" volumeName="kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391818 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391830 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391843 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391870 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3889db9a-bb9d-422e-a8a6-ffda0e64d867" volumeName="kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391883 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391894 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="803bdfd3-1098-4104-abc0-91f2b5ec5192" volumeName="kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391931 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="825cfc8b-20b7-481c-beaa-46a414be370a" volumeName="kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391945 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50e239d-68ef-4cde-88ab-ee15083433d2" volumeName="kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391958 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391970 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3889db9a-bb9d-422e-a8a6-ffda0e64d867" volumeName="kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391982 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c89b664a-dd5d-4631-bbae-d197c38825e2" volumeName="kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.391995 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392008 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392021 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392034 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caf92aa8-a824-4089-8fa8-9005b529346f" volumeName="kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392046 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392057 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392070 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="803bdfd3-1098-4104-abc0-91f2b5ec5192" volumeName="kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc" seLinuxMountContext="" Nov 04 02:07:02.392068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392082 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392094 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392106 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc9507ca-eeeb-4ed3-ab31-019380293e75" volumeName="kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392118 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d1f5f84-c79a-47fe-9b41-f6d8d7b10100" volumeName="kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392130 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392142 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c66eb7a2-b11d-422a-b75c-4dc43e4efe5d" volumeName="kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392155 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caf92aa8-a824-4089-8fa8-9005b529346f" volumeName="kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392167 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392179 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392191 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392204 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392217 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caf92aa8-a824-4089-8fa8-9005b529346f" volumeName="kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392231 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392252 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392265 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc9507ca-eeeb-4ed3-ab31-019380293e75" volumeName="kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392277 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392289 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392301 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392314 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50e239d-68ef-4cde-88ab-ee15083433d2" volumeName="kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392341 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392354 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392367 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392379 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ced39f9f-ae73-431b-aeb4-bb0b2ae2b218" volumeName="kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir" seLinuxMountContext="" Nov 04 02:07:02.393072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.392393 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.394001 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.393971 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.394056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394031 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.394056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394050 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls" seLinuxMountContext="" Nov 04 02:07:02.394148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394066 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e114adad-34e0-4007-9a4c-011c60df73c6" volumeName="kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf" seLinuxMountContext="" Nov 04 02:07:02.394148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394087 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config" seLinuxMountContext="" Nov 04 02:07:02.394148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394102 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 02:07:02.394148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394122 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 02:07:02.394148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394138 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d1f5f84-c79a-47fe-9b41-f6d8d7b10100" volumeName="kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394151 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394169 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394183 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394204 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394221 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394238 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394257 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e114adad-34e0-4007-9a4c-011c60df73c6" volumeName="kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394269 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394286 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394298 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394311 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30429b8d-d02a-4d8c-a500-a3bbf17e7a61" volumeName="kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394351 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394364 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394382 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.394393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394397 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ced39f9f-ae73-431b-aeb4-bb0b2ae2b218" volumeName="kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394412 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394432 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394447 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394471 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394485 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d1f5f84-c79a-47fe-9b41-f6d8d7b10100" volumeName="kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394500 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9082f258-87ca-4149-9e1e-69e87f0b2cb6" volumeName="kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394518 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394531 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394554 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394568 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b193ea-8136-4bb9-b98b-a3930e51f99c" volumeName="kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394585 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="825cfc8b-20b7-481c-beaa-46a414be370a" volumeName="kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394605 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9082f258-87ca-4149-9e1e-69e87f0b2cb6" volumeName="kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394620 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb61a6a8-f7ee-4140-8dff-1b292b2173b8" volumeName="kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394638 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66fdb210-e77b-4433-a0c3-0ac8262cdcb9" volumeName="kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394654 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394670 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b193ea-8136-4bb9-b98b-a3930e51f99c" volumeName="kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394690 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b193ea-8136-4bb9-b98b-a3930e51f99c" volumeName="kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394704 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394722 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394739 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394754 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d1f5f84-c79a-47fe-9b41-f6d8d7b10100" volumeName="kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394775 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb61a6a8-f7ee-4140-8dff-1b292b2173b8" volumeName="kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394829 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" volumeName="kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394843 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394863 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d1f5f84-c79a-47fe-9b41-f6d8d7b10100" volumeName="kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394876 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394896 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394910 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2e90242-f97c-45f4-abd7-4493c8d9cc42" volumeName="kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394922 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30429b8d-d02a-4d8c-a500-a3bbf17e7a61" volumeName="kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394940 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a78f67-2bc4-4d66-b09a-e93063641f64" volumeName="kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394954 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394975 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.394989 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50e239d-68ef-4cde-88ab-ee15083433d2" volumeName="kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395002 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0129d90-8271-46a8-b92b-e179632baf01" volumeName="kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395021 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" volumeName="kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls" seLinuxMountContext="" Nov 04 02:07:02.395034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395034 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83d93c1b-1835-43d7-88cf-371913654014" volumeName="kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395059 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30429b8d-d02a-4d8c-a500-a3bbf17e7a61" volumeName="kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395076 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66fdb210-e77b-4433-a0c3-0ac8262cdcb9" volumeName="kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395095 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66fdb210-e77b-4433-a0c3-0ac8262cdcb9" volumeName="kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395108 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395121 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ced39f9f-ae73-431b-aeb4-bb0b2ae2b218" volumeName="kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395139 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01c21249-e6ab-42b6-9391-a9f4ad402b8a" volumeName="kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395155 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395176 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7014e7c0-9d7d-4306-8e68-98437feeaf4e" volumeName="kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395190 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" volumeName="kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395204 6073 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a50529fc-30c2-4a6f-83db-f90819ad9a94" volumeName="kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config" seLinuxMountContext="" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395216 6073 reconstruct.go:97] "Volume reconstruction finished" Nov 04 02:07:02.396739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.395225 6073 reconciler.go:29] "Reconciler: start to sync state" Nov 04 02:07:02.416765 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.416695 6073 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 02:07:02.418361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.418324 6073 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 02:07:02.418499 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.418482 6073 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 02:07:02.418615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.418511 6073 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 02:07:02.418615 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.418556 6073 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 02:07:02.464826 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.464792 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" exitCode=0 Nov 04 02:07:02.470461 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.470413 6073 generic.go:356] "Generic (PLEG): container finished" podID="94607daadef9ea3ca6175efce35c3d4a" containerID="ab790b2418747a40376f998ae8b2b08b7a4d51e83e3e51e4818de08f7391ce93" exitCode=0 Nov 04 02:07:02.474532 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.474507 6073 generic.go:356] "Generic (PLEG): container finished" podID="9d57b05f-6e23-42fb-b3ff-25479d15b6ae" containerID="56112a0cb724e296eb374c27e2bf3a39cf5db71872bd856a15716183cb062825" exitCode=0 Nov 04 02:07:02.486936 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.486849 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4" exitCode=0 Nov 04 02:07:02.487592 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.487568 6073 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:07:02.490669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490640 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="e859ea26f2a692df63cc0a19b4c80b529c325bec1807bbf0c832b3d48cad90c4" exitCode=0 Nov 04 02:07:02.490669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490668 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="001619a6c6e1523db6e43396b3d146b06113af48c58dc98329f707b45d2d3348" exitCode=0 Nov 04 02:07:02.490845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490675 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="9a21ac436fc36befa309df209751a3d1b4c47968fc997424893211d8c266d67b" exitCode=0 Nov 04 02:07:02.490845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490683 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="e053aec983bc8a7e29ee760bba38cbf7520627369b6d7816c08392b77ee5d934" exitCode=0 Nov 04 02:07:02.490845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490690 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="4c01fa9cd28643c6b38025caf4db4964d123dd35b29467b607132710c81b8cb5" exitCode=0 Nov 04 02:07:02.490845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.490700 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50e239d-68ef-4cde-88ab-ee15083433d2" containerID="55b00b168ff0cd126f24415ccf0879291121721cb2f3a65f5f53a4bcf2992298" exitCode=0 Nov 04 02:07:02.518633 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.518604 6073 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 04 02:07:02.521829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.521807 6073 manager.go:324] Recovery completed Nov 04 02:07:02.546832 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.546809 6073 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:07:02.549976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.549958 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:07:02.550037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.549993 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:07:02.550037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.550004 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:07:02.553362 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553345 6073 cpu_manager.go:226] "Starting" policy="none" Nov 04 02:07:02.553362 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553358 6073 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 02:07:02.553509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553372 6073 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:07:02.553509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553374 6073 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 02:07:02.553594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553579 6073 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 02:07:02.553627 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553596 6073 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 02:07:02.553627 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553623 6073 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 02:07:02.553683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553629 6073 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 02:07:02.553683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553635 6073 policy_none.go:50] "Start" Nov 04 02:07:02.553683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553648 6073 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 02:07:02.553683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553658 6073 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 02:07:02.553786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553776 6073 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 02:07:02.553786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553783 6073 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 02:07:02.553841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.553789 6073 policy_none.go:44] "Start" Nov 04 02:07:02.554483 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.554465 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:07:02.554573 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.554497 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:07:02.554573 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.554509 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:07:02.558711 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.558679 6073 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 02:07:02.562283 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562266 6073 manager.go:341] "Starting Device Plugin manager" Nov 04 02:07:02.562394 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.562296 6073 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 02:07:02.562394 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562307 6073 server.go:87] "Starting device plugin registration server" Nov 04 02:07:02.562621 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562606 6073 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 02:07:02.562679 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562621 6073 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 02:07:02.562731 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562705 6073 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 02:07:02.562884 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562822 6073 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 02:07:02.562884 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562864 6073 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 02:07:02.562884 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562871 6073 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 02:07:02.563520 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.562924 6073 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:07:02.562853717Z","UUID":"e64fbbee-13bc-421a-abd1-4af2a8c0329e","Handler":null,"Name":"","Endpoint":""} Nov 04 02:07:02.564672 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.564650 6073 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 02:07:02.564756 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.564696 6073 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-45.us-west-1.compute.internal\" not found" Nov 04 02:07:02.564852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.564830 6073 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 02:07:02.564904 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.564870 6073 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 02:07:02.585806 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.585785 6073 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:07:02.585806 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.585801 6073 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:07:02.587284 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.587264 6073 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 02:07:02.632268 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.632234 6073 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:07:02.663321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.663295 6073 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 02:07:02.665751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.665734 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 02:07:02.665817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.665765 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 02:07:02.665817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.665775 6073 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-45.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 02:07:02.665895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.665831 6073 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.682115 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.682086 6073 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 02:07:02.719085 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719032 6073 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal"] Nov 04 02:07:02.719750 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719727 6073 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.719810 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719743 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"33ceb3d0c15f7c179f81071c0e8686a7","Type":"ContainerStarted","Data":"13fb33366fb6ecf6849f03da50e2f95ba14dabe67902d7e7a422e6b9bed21032"} Nov 04 02:07:02.719810 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719790 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"33ceb3d0c15f7c179f81071c0e8686a7","Type":"ContainerStarted","Data":"da864d85c3f2b57b1ad2d3da1e138e586fc2c6db997368c4e1386238eddc28b2"} Nov 04 02:07:02.719895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719810 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerStarted","Data":"31b3ed88aff40fa8ece592ac216c031d62dfb33c713b635f28b35f8c9292cd3c"} Nov 04 02:07:02.719895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719829 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerDied","Data":"ab790b2418747a40376f998ae8b2b08b7a4d51e83e3e51e4818de08f7391ce93"} Nov 04 02:07:02.719895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719848 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" event={"ID":"94607daadef9ea3ca6175efce35c3d4a","Type":"ContainerStarted","Data":"02f83d17321f947b39c2152825ad31a51a7922dfaf8eb7a6b854982949cabb46"} Nov 04 02:07:02.720003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.719989 6073 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.780672 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.780652 6073 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:07:02.780790 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.780694 6073 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.781511 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.781495 6073 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:07:02.781545 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:02.781532 6073 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.792211 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.792193 6073 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.792267 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.792253 6073 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.889242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.889210 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.889242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.889243 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.889465 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.889263 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989505 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989538 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989554 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989607 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989613 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94607daadef9ea3ca6175efce35c3d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"94607daadef9ea3ca6175efce35c3d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:02.989743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:02.989618 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33ceb3d0c15f7c179f81071c0e8686a7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" (UID: \"33ceb3d0c15f7c179f81071c0e8686a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:03.370168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.370124 6073 apiserver.go:52] "Watching apiserver" Nov 04 02:07:03.494880 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.494854 6073 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:03.495055 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.494981 6073 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:03.523532 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.523493 6073 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 02:07:03.524204 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.524184 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2","openshift-image-registry/node-ca-2gh6q","kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-56kwr","openshift-monitoring/metrics-server-db9bb74dd-5bnpg","openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg","openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv","openshift-console/console-7c76cc68f9-dcjvf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t","openshift-console/console-d8cf5d56b-xqwgh","openshift-dns/dns-default-gr2cj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal","openshift-ingress-canary/ingress-canary-mcj49","openshift-monitoring/prometheus-k8s-0","kube-system/global-pull-secret-syncer-tjxq6","openshift-cluster-node-tuning-operator/tuned-pfcpv","openshift-multus/network-metrics-daemon-5h9d5","openshift-multus/multus-additional-cni-plugins-z7kbm","openshift-network-operator/iptables-alerter-shp2z","openshift-monitoring/node-exporter-shpnc","openshift-insights/insights-runtime-extractor-7fl4d","openshift-monitoring/alertmanager-main-0","openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp","openshift-ovn-kubernetes/ovnkube-node-t2htq","kube-system/konnectivity-agent-g5dnk","openshift-multus/multus-ghkfg","openshift-dns/node-resolver-b55hv","openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4","openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr"] Nov 04 02:07:03.556214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.556190 6073 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:07:03.556377 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:03.556251 6073 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:03.556807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.556788 6073 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 02:07:03.556881 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:03.556836 6073 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.us-west-1.compute.internal" Nov 04 02:07:03.577612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577583 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.577612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577585 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.577763 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577661 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.577763 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577673 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.577763 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577663 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.577763 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577702 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.577941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577927 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.577992 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577931 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.578039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.577999 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.578039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.578002 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 02:07:03.578296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.578283 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 02:07:03.578349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.578294 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.578349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.578296 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 02:07:03.578466 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.578454 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 02:07:03.579446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579344 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 02:07:03.579446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579325 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 02:07:03.579446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579397 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kn9lr\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579473 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579488 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8fdm8\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579526 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ckptg\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579532 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579569 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579621 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579632 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-szjlx\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579682 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579701 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579709 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.579759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579748 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579794 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.579992 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qq5vw\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580044 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rtznt\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580080 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580110 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h9jgb\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580138 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h6pb6\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580203 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-db624\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580209 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.580288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580252 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z899q\"" Nov 04 02:07:03.580575 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.580540 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 02:07:03.583886 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.583872 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 02:07:03.584113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.584101 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ghp2x\"" Nov 04 02:07:03.584172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.584160 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 02:07:03.590851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.590835 6073 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 02:07:03.593596 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593580 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.593637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593607 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.593637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593625 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvjf\" (UniqueName: \"kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.593708 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593641 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.593708 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593698 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmnhc\" (UniqueName: \"kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.593766 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593721 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0129d90-8271-46a8-b92b-e179632baf01-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.593798 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593774 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.593863 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593849 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.593896 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593871 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.593947 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593933 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.593985 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593957 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.594014 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.593983 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.594043 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594011 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.594043 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594037 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.594100 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594061 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.594128 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594103 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.594157 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594135 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.594186 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594168 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:07:03.594217 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594196 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.594247 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594218 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.594247 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594240 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.594321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594261 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.594321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594276 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.594321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594291 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.594321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594308 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594317 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e90242-f97c-45f4-abd7-4493c8d9cc42-tmp\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594356 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594395 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594399 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594422 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594479 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-tmp\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594502 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.594528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594515 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc9507ca-eeeb-4ed3-ab31-019380293e75-konnectivity-ca\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594530 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594551 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594569 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594585 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594601 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594617 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594632 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594701 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.594785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594734 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594797 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594826 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594821 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a78f67-2bc4-4d66-b09a-e93063641f64-audit-log\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594848 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7g5l\" (UniqueName: \"kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.594969 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.595041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595019 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.595195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595051 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595074 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595133 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnczh\" (UniqueName: \"kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595165 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.595300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595196 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.595300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595212 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.595300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595230 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.595300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595256 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.595300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595282 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjq8c\" (UniqueName: \"kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.595476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595311 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.595476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595361 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.595476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595394 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rp4\" (UniqueName: \"kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.595476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595424 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.595476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595449 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.595612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595483 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.595612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595562 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.595612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595578 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.595612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595585 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.595717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595613 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.595717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595647 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.595717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595664 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.595717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595680 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.595717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595695 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595720 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595749 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595755 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595779 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhts\" (UniqueName: \"kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595796 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595811 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.595850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595837 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595872 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595900 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595918 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95zw\" (UniqueName: \"kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595961 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnzrd\" (UniqueName: \"kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.595987 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596012 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:07:03.596039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596037 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596061 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596084 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596107 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596132 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plqx\" (UniqueName: \"kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596161 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.596266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596188 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.596484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596280 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.596484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596320 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.596484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596365 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.596484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596452 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596493 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596537 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596568 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596650 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596655 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/803bdfd3-1098-4104-abc0-91f2b5ec5192-tmp-dir\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.596671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596667 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596668 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596706 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596733 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596763 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596790 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596823 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596846 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596850 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596904 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596948 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.596953 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597002 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597031 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597056 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cni-binary-copy\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.597056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597063 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597181 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597202 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597231 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597207 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597237 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597382 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597392 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597385 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kvkfc\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597382 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597434 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzhlz\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597450 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-original-pull-secret\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597298 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597528 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7wvp\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597538 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zd6qr\"" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597560 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597597 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597629 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.597707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597664 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597726 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597761 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597788 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-env-overrides\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597808 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597906 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597923 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597947 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597965 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597981 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.597997 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovn-node-metrics-cert\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598008 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598048 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598046 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598078 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598117 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598141 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598157 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598177 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjl6g\" (UniqueName: \"kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598197 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598217 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxmm2\" (UniqueName: \"kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598238 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598255 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598271 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24rv\" (UniqueName: \"kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598287 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.598298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598304 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598325 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxlk6\" (UniqueName: \"kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598371 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598399 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598474 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-config\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598544 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598562 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-iptables-alerter-script\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598591 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598642 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598663 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598680 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598702 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598733 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzdhz\" (UniqueName: \"kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598760 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598756 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-textfile\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598788 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598857 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-tmp-dir\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598902 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598926 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598944 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598960 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.598984 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599022 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzdg\" (UniqueName: \"kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599052 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599060 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-tuned\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599079 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599095 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e114adad-34e0-4007-9a4c-011c60df73c6-serviceca\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599105 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599171 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.599242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599220 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljpv\" (UniqueName: \"kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599262 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599224 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-daemon-config\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599294 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599359 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc9507ca-eeeb-4ed3-ab31-019380293e75-agent-certs\") pod \"konnectivity-agent-g5dnk\" (UID: \"fc9507ca-eeeb-4ed3-ab31-019380293e75\") " pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599344 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599429 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599459 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599491 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599519 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599594 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599577 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599632 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599651 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599667 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599683 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599699 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599714 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599766 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599782 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599799 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599816 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599832 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599847 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599856 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599868 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599863 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-ovnkube-script-lib\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599884 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-metrics-tls\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599988 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b193ea-8136-4bb9-b98b-a3930e51f99c-data-volume\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.599996 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600030 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600061 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600090 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600120 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st24x\" (UniqueName: \"kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600153 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600182 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600204 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600221 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600237 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.600426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600253 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600721 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600760 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3889db9a-bb9d-422e-a8a6-ffda0e64d867-cert\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600777 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600779 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082f258-87ca-4149-9e1e-69e87f0b2cb6-metrics-certs\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600844 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600900 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.600962 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-config-volume\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.601048 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.601086 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d78kv\" (UniqueName: \"kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.601118 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.601142 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.602052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.601210 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.605446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.605427 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a50e239d-68ef-4cde-88ab-ee15083433d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.606251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.606234 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70b193ea-8136-4bb9-b98b-a3930e51f99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.624619 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.624530 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 02:07:03.624619 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.624552 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 02:07:03.625316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625300 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.625316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625307 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 02:07:03.625421 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625407 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 02:07:03.625567 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625554 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 02:07:03.625605 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625566 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 02:07:03.625605 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625572 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 02:07:03.625947 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625934 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 02:07:03.625998 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.625946 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fjjw2\"" Nov 04 02:07:03.627925 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.627907 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 02:07:03.628861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.628845 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.629488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.629470 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Nov 04 02:07:03.630450 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.630435 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.630544 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.630529 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.630585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.630551 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.631636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.631621 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.634344 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.634317 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.634401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.634350 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.635547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.635528 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.635585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.635537 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.636752 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.636735 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.652283 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.652261 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvjf\" (UniqueName: \"kubernetes.io/projected/e114adad-34e0-4007-9a4c-011c60df73c6-kube-api-access-nhvjf\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.657464 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.657440 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmnhc\" (UniqueName: \"kubernetes.io/projected/803bdfd3-1098-4104-abc0-91f2b5ec5192-kube-api-access-mmnhc\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.657769 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.657752 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnczh\" (UniqueName: \"kubernetes.io/projected/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-kube-api-access-jnczh\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.658131 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.658116 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhts\" (UniqueName: \"kubernetes.io/projected/ced39f9f-ae73-431b-aeb4-bb0b2ae2b218-kube-api-access-kwhts\") pod \"dns-default-gr2cj\" (UID: \"ced39f9f-ae73-431b-aeb4-bb0b2ae2b218\") " pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:03.659608 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.659593 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jzz\" (UniqueName: \"kubernetes.io/projected/c89b664a-dd5d-4631-bbae-d197c38825e2-kube-api-access-l4jzz\") pod \"network-check-target-56kwr\" (UID: \"c89b664a-dd5d-4631-bbae-d197c38825e2\") " pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:07:03.662954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.662938 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnzrd\" (UniqueName: \"kubernetes.io/projected/a50e239d-68ef-4cde-88ab-ee15083433d2-kube-api-access-bnzrd\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.666702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.666685 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxmm2\" (UniqueName: \"kubernetes.io/projected/72d28c97-8c3a-4ccf-8156-c496da17cbea-kube-api-access-fxmm2\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.666847 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.666827 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxlk6\" (UniqueName: \"kubernetes.io/projected/9082f258-87ca-4149-9e1e-69e87f0b2cb6-kube-api-access-fxlk6\") pod \"network-metrics-daemon-5h9d5\" (UID: \"9082f258-87ca-4149-9e1e-69e87f0b2cb6\") " pod="openshift-multus/network-metrics-daemon-5h9d5" Nov 04 02:07:03.666883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.666858 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjl6g\" (UniqueName: \"kubernetes.io/projected/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-api-access-kjl6g\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.668190 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.668168 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st24x\" (UniqueName: \"kubernetes.io/projected/3889db9a-bb9d-422e-a8a6-ffda0e64d867-kube-api-access-st24x\") pod \"ingress-canary-mcj49\" (UID: \"3889db9a-bb9d-422e-a8a6-ffda0e64d867\") " pod="openshift-ingress-canary/ingress-canary-mcj49" Nov 04 02:07:03.668346 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.668311 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljpv\" (UniqueName: \"kubernetes.io/projected/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-kube-api-access-2ljpv\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.668474 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.668459 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzdg\" (UniqueName: \"kubernetes.io/projected/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-kube-api-access-ddzdg\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.668530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.668460 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24rv\" (UniqueName: \"kubernetes.io/projected/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-kube-api-access-b24rv\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.678551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678475 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 02:07:03.678551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678539 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 02:07:03.678551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678543 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Nov 04 02:07:03.678717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678546 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 02:07:03.678717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678585 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 02:07:03.678717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678602 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 02:07:03.678717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678666 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 02:07:03.678717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678706 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 02:07:03.678860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.678796 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.679404 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679389 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.679517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679499 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 02:07:03.679591 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679524 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 02:07:03.679637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679593 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 02:07:03.679637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679620 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679919 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679928 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679948 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679972 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.679990 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680017 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680018 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Nov 04 02:07:03.680079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680042 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xlwps\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680222 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jsvqb\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680252 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680271 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qgk7m\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680275 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fh89s\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680257 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680292 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680259 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680280 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680292 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680361 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680419 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680476 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680523 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680528 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680542 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7kg86\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680543 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680554 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680579 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680594 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680617 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680617 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680630 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6hm7q\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680655 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 02:07:03.680702 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680663 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680746 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2iitb07vi68ac\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680747 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680774 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680788 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680815 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680824 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680835 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680789 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680860 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sd8ds\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680875 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-26iig1f3jpl7f\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680887 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.680988 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-metrics-server-audit-profiles\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681095 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681115 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-w9wdz\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681139 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8lp1mckvjcva\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681223 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681230 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681248 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lnjv7\"" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681497 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.681945 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.681507 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.682613 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.682007 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzdhz\" (UniqueName: \"kubernetes.io/projected/e2e90242-f97c-45f4-abd7-4493c8d9cc42-kube-api-access-vzdhz\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.682613 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.682254 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 02:07:03.682695 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.682681 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 02:07:03.682768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.682695 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 02:07:03.683172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.683158 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95zw\" (UniqueName: \"kubernetes.io/projected/825cfc8b-20b7-481c-beaa-46a414be370a-kube-api-access-n95zw\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:07:03.684526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684512 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.684591 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684547 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.684591 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684580 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0129d90-8271-46a8-b92b-e179632baf01-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.684701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684665 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.684701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684672 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.684701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684678 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.684701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684688 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.684825 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684701 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.684825 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684772 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.684825 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684811 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.684912 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684844 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.684912 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684851 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.684912 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.684895 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0129d90-8271-46a8-b92b-e179632baf01-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.685068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.685054 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.685963 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.685946 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686013 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.685974 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686013 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.685992 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.686013 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.685994 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-tls\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.686161 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686082 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/825cfc8b-20b7-481c-beaa-46a414be370a-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fw6nr\" (UID: \"825cfc8b-20b7-481c-beaa-46a414be370a\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:07:03.686213 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686178 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686213 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686200 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-tls\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.686285 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686215 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.686285 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686215 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686285 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686237 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-secret-telemeter-client\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.686285 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686216 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.686285 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686237 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686366 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-client-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.686530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686367 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-metrics-client-ca\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.686530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686380 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.686530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686416 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686475 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.686789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686620 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686864 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686849 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7g5l\" (UniqueName: \"kubernetes.io/projected/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-kube-api-access-c7g5l\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.686933 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686917 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.686988 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686967 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.687038 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.686999 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.687038 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687026 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.687118 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687048 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.687394 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687375 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.687513 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687499 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjq8c\" (UniqueName: \"kubernetes.io/projected/40a78f67-2bc4-4d66-b09a-e93063641f64-kube-api-access-tjq8c\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.687544 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687515 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-federate-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.687919 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.687904 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.688055 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688041 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70b193ea-8136-4bb9-b98b-a3930e51f99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.688188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688172 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.688188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688180 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.688297 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688196 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"console-7c76cc68f9-dcjvf\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:03.688369 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688343 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7014e7c0-9d7d-4306-8e68-98437feeaf4e-telemeter-client-tls\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.688440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688419 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"console-75dd9456cc-2tsv2\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:03.688440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688426 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2e90242-f97c-45f4-abd7-4493c8d9cc42-secret-grpc-tls\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.688619 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688600 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.688650 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688633 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78kv\" (UniqueName: \"kubernetes.io/projected/7014e7c0-9d7d-4306-8e68-98437feeaf4e-kube-api-access-d78kv\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.688684 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688658 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rp4\" (UniqueName: \"kubernetes.io/projected/d0129d90-8271-46a8-b92b-e179632baf01-kube-api-access-78rp4\") pod \"kube-state-metrics-5dc8d7d89b-52mz4\" (UID: \"d0129d90-8271-46a8-b92b-e179632baf01\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-52mz4" Nov 04 02:07:03.688714 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688662 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.688764 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688747 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7014e7c0-9d7d-4306-8e68-98437feeaf4e-serving-certs-ca-bundle\") pod \"telemeter-client-5744cbb8b4-tmdfp\" (UID: \"7014e7c0-9d7d-4306-8e68-98437feeaf4e\") " pod="openshift-monitoring/telemeter-client-5744cbb8b4-tmdfp" Nov 04 02:07:03.688845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.688821 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a78f67-2bc4-4d66-b09a-e93063641f64-secret-metrics-server-client-certs\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.689023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689008 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plqx\" (UniqueName: \"kubernetes.io/projected/caf92aa8-a824-4089-8fa8-9005b529346f-kube-api-access-2plqx\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.689139 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689127 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-metrics-client-ca\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.689244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689233 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a78f67-2bc4-4d66-b09a-e93063641f64-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9bb74dd-5bnpg\" (UID: \"40a78f67-2bc4-4d66-b09a-e93063641f64\") " pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:03.689299 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689284 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"console-d8cf5d56b-xqwgh\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:03.689351 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689317 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.689995 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689982 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf92aa8-a824-4089-8fa8-9005b529346f-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.690032 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.689988 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.690062 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.690028 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"alertmanager-main-0\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:03.690105 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.690092 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.690135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.690095 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf92aa8-a824-4089-8fa8-9005b529346f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hp6sv\" (UID: \"caf92aa8-a824-4089-8fa8-9005b529346f\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hp6sv" Nov 04 02:07:03.691219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.691206 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:03.691312 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.691299 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2e90242-f97c-45f4-abd7-4493c8d9cc42-metrics-client-ca\") pod \"thanos-querier-764b8c9c9c-hkjbg\" (UID: \"e2e90242-f97c-45f4-abd7-4493c8d9cc42\") " pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:03.702655 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702631 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.702743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702659 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.702743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702675 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.702743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702689 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.702743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702706 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.702743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702736 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702754 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70b193ea-8136-4bb9-b98b-a3930e51f99c-crio-socket\") pod \"insights-runtime-extractor-7fl4d\" (UID: \"70b193ea-8136-4bb9-b98b-a3930e51f99c\") " pod="openshift-insights/insights-runtime-extractor-7fl4d" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702790 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-log-socket\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702790 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-device-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702801 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702835 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702827 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-lib-modules\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702764 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-hostroot\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702860 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysconfig\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702874 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702893 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702914 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702930 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-conf-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702940 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.702955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702959 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-system-cni-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702960 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-etc-selinux\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702965 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702988 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.702991 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb61a6a8-f7ee-4140-8dff-1b292b2173b8-host-slash\") pod \"iptables-alerter-shp2z\" (UID: \"bb61a6a8-f7ee-4140-8dff-1b292b2173b8\") " pod="openshift-network-operator/iptables-alerter-shp2z" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703021 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703028 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-bin\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703040 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-etc-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703093 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-etc-kubernetes\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703110 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703132 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-systemd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703144 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703173 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703196 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703221 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703231 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-sys-fs\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703248 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703255 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-kubelet-config\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703271 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-cni-netd\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703275 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703280 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c66eb7a2-b11d-422a-b75c-4dc43e4efe5d-dbus\") pod \"global-pull-secret-syncer-tjxq6\" (UID: \"c66eb7a2-b11d-422a-b75c-4dc43e4efe5d\") " pod="kube-system/global-pull-secret-syncer-tjxq6" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703308 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703316 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/803bdfd3-1098-4104-abc0-91f2b5ec5192-hosts-file\") pod \"node-resolver-b55hv\" (UID: \"803bdfd3-1098-4104-abc0-91f2b5ec5192\") " pod="openshift-dns/node-resolver-b55hv" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703321 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-node-log\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703363 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703389 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703385 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-k8s-cni-cncf-io\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703404 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703427 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703431 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-sys\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703451 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703459 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-cnibin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703469 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703476 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-slash\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703481 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-socket-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703489 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703550 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-systemd-units\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703555 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-node-exporter-wtmp\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703562 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.703572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703582 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703605 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703623 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703483 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-kubelet\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703649 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703672 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-host\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703676 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-kubelet\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703682 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703743 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-root\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703696 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703781 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-modprobe-d\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703786 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703810 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703692 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-netns\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703845 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-run-ovn\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703864 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703910 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703934 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.703962 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704016 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-systemd\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704040 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704066 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704089 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704090 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704098 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-sysctl-conf\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704106 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704134 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704145 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d57b05f-6e23-42fb-b3ff-25479d15b6ae-sys\") pod \"node-exporter-shpnc\" (UID: \"9d57b05f-6e23-42fb-b3ff-25479d15b6ae\") " pod="openshift-monitoring/node-exporter-shpnc" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704150 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704156 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-var-lib-kubelet\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704181 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72d28c97-8c3a-4ccf-8156-c496da17cbea-registration-dir\") pod \"aws-ebs-csi-driver-node-g2v9t\" (UID: \"72d28c97-8c3a-4ccf-8156-c496da17cbea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g2v9t" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704185 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704202 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-run\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704212 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704222 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704236 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704251 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66fdb210-e77b-4433-a0c3-0ac8262cdcb9-etc-kubernetes\") pod \"tuned-pfcpv\" (UID: \"66fdb210-e77b-4433-a0c3-0ac8262cdcb9\") " pod="openshift-cluster-node-tuning-operator/tuned-pfcpv" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704250 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704260 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704288 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704295 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-run-multus-certs\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704298 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-host-run-netns\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704319 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704350 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704371 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704376 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-multus-socket-dir-parent\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704390 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e114adad-34e0-4007-9a4c-011c60df73c6-host\") pod \"node-ca-2gh6q\" (UID: \"e114adad-34e0-4007-9a4c-011c60df73c6\") " pod="openshift-image-registry/node-ca-2gh6q" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704398 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704425 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d1f5f84-c79a-47fe-9b41-f6d8d7b10100-var-lib-openvswitch\") pod \"ovnkube-node-t2htq\" (UID: \"8d1f5f84-c79a-47fe-9b41-f6d8d7b10100\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704428 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704464 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-bin\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704464 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-host-var-lib-cni-multus\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704488 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704515 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704584 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a50e239d-68ef-4cde-88ab-ee15083433d2-cnibin\") pod \"multus-additional-cni-plugins-z7kbm\" (UID: \"a50e239d-68ef-4cde-88ab-ee15083433d2\") " pod="openshift-multus/multus-additional-cni-plugins-z7kbm" Nov 04 02:07:03.705083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:03.704629 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30429b8d-d02a-4d8c-a500-a3bbf17e7a61-system-cni-dir\") pod \"multus-ghkfg\" (UID: \"30429b8d-d02a-4d8c-a500-a3bbf17e7a61\") " pod="openshift-multus/multus-ghkfg" Nov 04 02:07:04.810937 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:04.810908 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:05.111138 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:05.111059 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:05.635497 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:05.635467 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gr2cj" Nov 04 02:07:05.726151 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:05.726107 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:05.726378 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:05.726359 6073 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 02:07:07.177015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:07.176987 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-764b8c9c9c-hkjbg" Nov 04 02:07:07.418958 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:07.418931 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-db9bb74dd-5bnpg" Nov 04 02:07:07.968194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:07.968164 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fw6nr" Nov 04 02:07:08.505633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:08.505609 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:12.800613 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:12.800522 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:07:16.569777 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:16.569749 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2htq" Nov 04 02:07:20.013639 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013608 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013872 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="alertmanager" containerID="cri-o://46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da" gracePeriod=120 Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013904 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy" containerID="cri-o://8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68" gracePeriod=120 Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013921 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="prom-label-proxy" containerID="cri-o://c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229" gracePeriod=120 Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013935 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-web" containerID="cri-o://52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde" gracePeriod=120 Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013961 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="config-reloader" containerID="cri-o://36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888" gracePeriod=120 Nov 04 02:07:20.014084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.013986 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-metric" containerID="cri-o://801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0" gracePeriod=120 Nov 04 02:07:20.563117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563083 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229" exitCode=0 Nov 04 02:07:20.563117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563107 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0" exitCode=0 Nov 04 02:07:20.563117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563114 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68" exitCode=0 Nov 04 02:07:20.563117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563120 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888" exitCode=0 Nov 04 02:07:20.563117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563125 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da" exitCode=0 Nov 04 02:07:20.563523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563164 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229"} Nov 04 02:07:20.563523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563249 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0"} Nov 04 02:07:20.563523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563287 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68"} Nov 04 02:07:20.563523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563324 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888"} Nov 04 02:07:20.563523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:20.563391 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da"} Nov 04 02:07:21.016070 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.016038 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7956cf9f8-hq7wl"] Nov 04 02:07:21.018765 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.018747 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.078574 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.078543 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7956cf9f8-hq7wl"] Nov 04 02:07:21.186036 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186003 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-console-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186058 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-trusted-ca-bundle\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186091 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186227 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186165 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfmdk\" (UniqueName: \"kubernetes.io/projected/8db77b28-1e54-494a-85c6-d09d6abdd724-kube-api-access-tfmdk\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186227 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186218 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-oauth-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186266 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-service-ca\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186320 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186296 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db77b28-1e54-494a-85c6-d09d6abdd724-tmp\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.186392 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.186324 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-oauth-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.276800 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.276741 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:21.286975 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.286956 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-service-ca\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287051 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.286985 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db77b28-1e54-494a-85c6-d09d6abdd724-tmp\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287051 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287012 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-oauth-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287051 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287041 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-console-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287062 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-trusted-ca-bundle\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287087 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287141 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfmdk\" (UniqueName: \"kubernetes.io/projected/8db77b28-1e54-494a-85c6-d09d6abdd724-kube-api-access-tfmdk\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287191 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-oauth-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287456 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8db77b28-1e54-494a-85c6-d09d6abdd724-tmp\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287483 6073 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 02:07:21.287760 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287739 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-service-ca\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.287991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.287964 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-console-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.288099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.288017 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-oauth-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.288241 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.288224 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db77b28-1e54-494a-85c6-d09d6abdd724-trusted-ca-bundle\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.289801 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.289781 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-serving-cert\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.289897 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.289881 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db77b28-1e54-494a-85c6-d09d6abdd724-console-oauth-config\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.348960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.348931 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfmdk\" (UniqueName: \"kubernetes.io/projected/8db77b28-1e54-494a-85c6-d09d6abdd724-kube-api-access-tfmdk\") pod \"console-7956cf9f8-hq7wl\" (UID: \"8db77b28-1e54-494a-85c6-d09d6abdd724\") " pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.388297 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388267 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388314 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388388 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388469 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388506 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388538 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388734 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388607 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388734 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388622 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:21.388734 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388666 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388734 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388715 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388751 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388825 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388880 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.388950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.388917 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") pod \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\" (UID: \"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37\") " Nov 04 02:07:21.389408 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.389387 6073 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-main-db\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.391584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.391556 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:21.391733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.391563 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:21.392018 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.391981 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.392167 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392141 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "kube-api-access-9j7gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:21.392240 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392218 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.392306 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392257 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:21.392391 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392314 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.392647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392626 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.392934 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.392908 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:21.393156 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.393141 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.397729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.397708 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.405211 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.405194 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config" pod "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" (UID: "e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:21.490608 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490576 6073 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-cluster-tls-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490608 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490605 6073 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-web-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490617 6073 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490627 6073 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-tls-assets\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490637 6073 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490646 6073 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-metrics-client-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490654 6073 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-main-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490663 6073 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-out\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490671 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j7gz\" (UniqueName: \"kubernetes.io/projected/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-kube-api-access-9j7gz\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490680 6073 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490690 6073 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-config-volume\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.490786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.490698 6073 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:21.570196 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.570113 6073 generic.go:356] "Generic (PLEG): container finished" podID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerID="52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde" exitCode=0 Nov 04 02:07:21.570196 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.570188 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 02:07:21.570441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.570197 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde"} Nov 04 02:07:21.570441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.570285 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37","Type":"ContainerDied","Data":"752921911cbca91f619ce4459741812321f0c9c7d02dfeb296a4cfb6185575b0"} Nov 04 02:07:21.570441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.570348 6073 scope.go:122] "RemoveContainer" containerID="c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229" Nov 04 02:07:21.579505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.579483 6073 scope.go:122] "RemoveContainer" containerID="801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0" Nov 04 02:07:21.586265 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.586249 6073 scope.go:122] "RemoveContainer" containerID="8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68" Nov 04 02:07:21.593236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.593221 6073 scope.go:122] "RemoveContainer" containerID="52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde" Nov 04 02:07:21.599759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.599743 6073 scope.go:122] "RemoveContainer" containerID="36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888" Nov 04 02:07:21.606729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.606713 6073 scope.go:122] "RemoveContainer" containerID="46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da" Nov 04 02:07:21.613568 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.613551 6073 scope.go:122] "RemoveContainer" containerID="46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4" Nov 04 02:07:21.620050 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620035 6073 scope.go:122] "RemoveContainer" containerID="c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229" Nov 04 02:07:21.620306 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.620289 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229\": container with ID starting with c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229 not found: ID does not exist" containerID="c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229" Nov 04 02:07:21.620399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620315 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229"} err="failed to get container status \"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229\": rpc error: code = NotFound desc = could not find container \"c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229\": container with ID starting with c03c9e995609ebb688dd76f129bd6746b58a1add0c96720a49d6ccc39f96c229 not found: ID does not exist" Nov 04 02:07:21.620399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620368 6073 scope.go:122] "RemoveContainer" containerID="801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0" Nov 04 02:07:21.620602 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.620588 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0\": container with ID starting with 801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0 not found: ID does not exist" containerID="801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0" Nov 04 02:07:21.620648 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620606 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0"} err="failed to get container status \"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0\": rpc error: code = NotFound desc = could not find container \"801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0\": container with ID starting with 801643ce6714360d072fc4a4015267208318ed9d4ea572bd6672a8b8acaf45b0 not found: ID does not exist" Nov 04 02:07:21.620648 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620618 6073 scope.go:122] "RemoveContainer" containerID="8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68" Nov 04 02:07:21.620838 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.620819 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68\": container with ID starting with 8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68 not found: ID does not exist" containerID="8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68" Nov 04 02:07:21.620877 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620847 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68"} err="failed to get container status \"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68\": rpc error: code = NotFound desc = could not find container \"8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68\": container with ID starting with 8c5bcce9177a4714b1cf21bb6358190d66be4cb9a912e915890b147a7d09ed68 not found: ID does not exist" Nov 04 02:07:21.620877 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.620863 6073 scope.go:122] "RemoveContainer" containerID="52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde" Nov 04 02:07:21.621083 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.621067 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde\": container with ID starting with 52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde not found: ID does not exist" containerID="52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde" Nov 04 02:07:21.621150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621093 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde"} err="failed to get container status \"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde\": rpc error: code = NotFound desc = could not find container \"52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde\": container with ID starting with 52acea509096bf2140c094af27fa44be0d744ab5c1859b1fe789b0e0113edcde not found: ID does not exist" Nov 04 02:07:21.621150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621114 6073 scope.go:122] "RemoveContainer" containerID="36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888" Nov 04 02:07:21.621391 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.621375 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888\": container with ID starting with 36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888 not found: ID does not exist" containerID="36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888" Nov 04 02:07:21.621434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621395 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888"} err="failed to get container status \"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888\": rpc error: code = NotFound desc = could not find container \"36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888\": container with ID starting with 36a26b07079c92ba2f8840ae974ac7219d908d3e64619038dc906c48d321e888 not found: ID does not exist" Nov 04 02:07:21.621434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621407 6073 scope.go:122] "RemoveContainer" containerID="46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da" Nov 04 02:07:21.621633 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.621617 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da\": container with ID starting with 46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da not found: ID does not exist" containerID="46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da" Nov 04 02:07:21.621673 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621638 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da"} err="failed to get container status \"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da\": rpc error: code = NotFound desc = could not find container \"46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da\": container with ID starting with 46088d89523da8e6360170ef2e61d680ef823d20aa3a07d931e5dcfc75b227da not found: ID does not exist" Nov 04 02:07:21.621673 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621652 6073 scope.go:122] "RemoveContainer" containerID="46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4" Nov 04 02:07:21.621857 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:21.621841 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4\": container with ID starting with 46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4 not found: ID does not exist" containerID="46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4" Nov 04 02:07:21.621905 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.621863 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4"} err="failed to get container status \"46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4\": rpc error: code = NotFound desc = could not find container \"46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4\": container with ID starting with 46b2fefcdf27b121992eaecbf3ec432e4ff6bdcc281a8d2aac0ca4c66e91c2a4 not found: ID does not exist" Nov 04 02:07:21.629042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.629018 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:21.742664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.742638 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:07:21.746834 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.746811 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 02:07:21.998398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:21.998304 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7956cf9f8-hq7wl"] Nov 04 02:07:22.001507 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:22.001483 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db77b28_1e54_494a_85c6_d09d6abdd724.slice/crio-0113fff9d0f2da3267c9923b34f863f7d1cafb3af50a785d8523895ad3db3ef5 WatchSource:0}: Error finding container 0113fff9d0f2da3267c9923b34f863f7d1cafb3af50a785d8523895ad3db3ef5: Status 404 returned error can't find the container with id 0113fff9d0f2da3267c9923b34f863f7d1cafb3af50a785d8523895ad3db3ef5 Nov 04 02:07:22.423973 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:22.423933 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" path="/var/lib/kubelet/pods/e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37/volumes" Nov 04 02:07:22.575956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:22.575913 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956cf9f8-hq7wl" event={"ID":"8db77b28-1e54-494a-85c6-d09d6abdd724","Type":"ContainerStarted","Data":"df196105d0e3a339743d404c19a1aaa79ee54a3f4f2d14ad4bb2b99d15844513"} Nov 04 02:07:22.576146 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:22.575970 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956cf9f8-hq7wl" event={"ID":"8db77b28-1e54-494a-85c6-d09d6abdd724","Type":"ContainerStarted","Data":"0113fff9d0f2da3267c9923b34f863f7d1cafb3af50a785d8523895ad3db3ef5"} Nov 04 02:07:22.693453 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:22.693325 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7956cf9f8-hq7wl" podStartSLOduration=2.693307166 podStartE2EDuration="2.693307166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:22.692743854 +0000 UTC m=+20.388434058" watchObservedRunningTime="2026-11-04 02:07:22.693307166 +0000 UTC m=+20.388997368" Nov 04 02:07:23.856774 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:23.856736 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:07:25.815000 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.814966 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:25.815408 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815326 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="prometheus" containerID="cri-o://1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" gracePeriod=600 Nov 04 02:07:25.815408 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815363 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy" containerID="cri-o://51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" gracePeriod=600 Nov 04 02:07:25.815537 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815404 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-web" containerID="cri-o://b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" gracePeriod=600 Nov 04 02:07:25.815537 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815454 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-thanos" containerID="cri-o://22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" gracePeriod=600 Nov 04 02:07:25.815537 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815403 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="thanos-sidecar" containerID="cri-o://2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" gracePeriod=600 Nov 04 02:07:25.815537 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:25.815518 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="config-reloader" containerID="cri-o://51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" gracePeriod=600 Nov 04 02:07:26.105320 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.105299 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.136588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136565 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136608 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136641 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136676 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136704 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136724 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136749 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136777 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.136850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136825 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136851 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136890 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136926 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136957 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.136987 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137023 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137063 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137065 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.137458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137122 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137195 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137234 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137268 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") pod \"83d93c1b-1835-43d7-88cf-371913654014\" (UID: \"83d93c1b-1835-43d7-88cf-371913654014\") " Nov 04 02:07:26.137690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137610 6073 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.137690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137643 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.138154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137872 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:26.138154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.137903 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.138960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.138731 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.138960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.138906 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.139110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.139025 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.139110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.139064 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:26.140122 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.140096 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.140623 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.140560 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.141315 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.141290 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.141428 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.141360 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:26.141716 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.141675 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:26.141799 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.141754 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.141864 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.141843 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.142052 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.142028 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.142114 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.142098 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.142149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.142139 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.142560 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.142539 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "kube-api-access-lljqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:26.152400 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.152379 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config" pod "83d93c1b-1835-43d7-88cf-371913654014" (UID: "83d93c1b-1835-43d7-88cf-371913654014"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:26.238570 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238535 6073 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238570 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238571 6073 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238586 6073 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238600 6073 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-kube-rbac-proxy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238616 6073 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238629 6073 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238643 6073 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238655 6073 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-web-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238670 6073 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238686 6073 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238700 6073 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238713 6073 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-config-out\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238726 6073 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d93c1b-1835-43d7-88cf-371913654014-prometheus-k8s-db\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238740 6073 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238754 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lljqf\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-kube-api-access-lljqf\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238767 6073 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d93c1b-1835-43d7-88cf-371913654014-configmap-metrics-client-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238781 6073 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-metrics-client-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238794 6073 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d93c1b-1835-43d7-88cf-371913654014-secret-grpc-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.238814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.238807 6073 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d93c1b-1835-43d7-88cf-371913654014-tls-assets\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:26.593872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593830 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" exitCode=0 Nov 04 02:07:26.593872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593869 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" exitCode=0 Nov 04 02:07:26.593872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593877 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" exitCode=0 Nov 04 02:07:26.593872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593882 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" exitCode=0 Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593889 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" exitCode=0 Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593897 6073 generic.go:356] "Generic (PLEG): container finished" podID="83d93c1b-1835-43d7-88cf-371913654014" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" exitCode=0 Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593907 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593919 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.593999 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594028 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594059 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594067 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.594119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594098 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} Nov 04 02:07:26.594526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594135 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} Nov 04 02:07:26.594526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.594177 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d93c1b-1835-43d7-88cf-371913654014","Type":"ContainerDied","Data":"b76f0b1a5087d14634731f7484fffeb9179163497163f28c23f75aa18f35ded9"} Nov 04 02:07:26.601780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.601761 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.608566 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.608550 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.614817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.614802 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.621260 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.621243 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.627723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.627701 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.634681 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.634662 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.641191 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641174 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.641494 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.641477 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.641555 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641505 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.641555 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641522 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.641759 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.641746 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.641836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641762 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.641836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641775 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.641956 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.641942 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.641992 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641959 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.641992 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.641971 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.642184 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.642167 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.642220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642191 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.642220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642208 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.642468 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.642449 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.642509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642475 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.642509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642491 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.642716 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.642702 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.642757 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642719 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.642757 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642731 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.642944 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:26.642928 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.642988 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642948 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.642988 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.642959 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.643175 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643158 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.643219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643177 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.643391 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643375 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.643446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643391 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.643616 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643597 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.643661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643616 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.643807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643790 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.643849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.643807 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.644033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644009 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.644033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644031 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.644263 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644245 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.644352 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644264 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.644548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644530 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.644595 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644550 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.644769 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644752 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.644845 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644770 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.644997 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644981 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.645041 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.644998 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.645209 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645192 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.645263 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645210 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.645437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645420 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.645437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645436 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.645642 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645624 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.645709 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645642 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.645859 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645843 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.645900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.645859 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.646106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646087 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.646180 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646108 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.646382 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646350 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.646382 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646379 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.646652 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646632 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.646731 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646664 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.646938 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646918 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.646986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.646939 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.647148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647130 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.647193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647149 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.647431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647409 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.647431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647431 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.647647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647632 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.647686 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647647 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.647862 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647845 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.647901 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.647864 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.648092 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648074 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.648150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648093 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.648347 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648313 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.648403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648367 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.648610 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648591 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.648610 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648609 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.648837 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648820 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.648893 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.648837 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.649044 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649030 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.649093 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649045 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.649257 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649241 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.649294 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649257 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.649487 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649470 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.649534 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649487 6073 scope.go:122] "RemoveContainer" containerID="22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842" Nov 04 02:07:26.649706 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649691 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842"} err="failed to get container status \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": rpc error: code = NotFound desc = could not find container \"22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842\": container with ID starting with 22873288b130255de59633a573562f5386e6aca0f336e1f4f31be13654881842 not found: ID does not exist" Nov 04 02:07:26.649748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649707 6073 scope.go:122] "RemoveContainer" containerID="51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f" Nov 04 02:07:26.649903 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649885 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f"} err="failed to get container status \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": rpc error: code = NotFound desc = could not find container \"51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f\": container with ID starting with 51dd913890749e93eb115ea86757cef9083cdd1b80cac24ba0cea2faaa70709f not found: ID does not exist" Nov 04 02:07:26.649952 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.649903 6073 scope.go:122] "RemoveContainer" containerID="b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2" Nov 04 02:07:26.650110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650094 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2"} err="failed to get container status \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": rpc error: code = NotFound desc = could not find container \"b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2\": container with ID starting with b5049032c626f7a63fa1965c5bc565d9b7cb89c7fa2e8fa6aae72f48c58a30c2 not found: ID does not exist" Nov 04 02:07:26.650110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650108 6073 scope.go:122] "RemoveContainer" containerID="2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d" Nov 04 02:07:26.650319 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650303 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d"} err="failed to get container status \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": rpc error: code = NotFound desc = could not find container \"2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d\": container with ID starting with 2d3ab20d6239bb4af25fc821c23aae218517a9df9a50f6c3722aef473662758d not found: ID does not exist" Nov 04 02:07:26.650406 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650320 6073 scope.go:122] "RemoveContainer" containerID="51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad" Nov 04 02:07:26.650554 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650536 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad"} err="failed to get container status \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": rpc error: code = NotFound desc = could not find container \"51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad\": container with ID starting with 51a434893a0500fd1134c545b37a280c72aeff44b3f9eece19944d4f5c461fad not found: ID does not exist" Nov 04 02:07:26.650554 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650553 6073 scope.go:122] "RemoveContainer" containerID="1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d" Nov 04 02:07:26.650754 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650738 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d"} err="failed to get container status \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": rpc error: code = NotFound desc = could not find container \"1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d\": container with ID starting with 1919eb93f399bdab7dc23347a98099fef0f80d9fec85a1a25a03b63d01041d4d not found: ID does not exist" Nov 04 02:07:26.650754 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650753 6073 scope.go:122] "RemoveContainer" containerID="1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa" Nov 04 02:07:26.650951 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.650936 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa"} err="failed to get container status \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": rpc error: code = NotFound desc = could not find container \"1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa\": container with ID starting with 1db8b7becb629232ac692cbfe2632c1b3f5c7c5ca78cfcbece1e2cbe2d7f54fa not found: ID does not exist" Nov 04 02:07:26.759512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.759433 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:26.765710 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.765691 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:26.808603 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808528 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808926 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808945 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808958 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-thanos" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808964 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-thanos" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808972 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="prometheus" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808978 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="prometheus" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808985 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="alertmanager" Nov 04 02:07:26.808989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808990 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="alertmanager" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.808999 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809004 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809009 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-metric" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809016 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-metric" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809025 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809032 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809048 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809052 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809059 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809063 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809070 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="thanos-sidecar" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809074 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="thanos-sidecar" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809080 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809083 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809092 6073 cpu_manager.go:414] "containerMap: removing container" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809096 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809103 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="prom-label-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809107 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="prom-label-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809120 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809125 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809171 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-thanos" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809178 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="prometheus" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809183 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="alertmanager" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809190 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809196 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-metric" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809202 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="init-config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809208 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809212 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809217 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="thanos-sidecar" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809221 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809226 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="config-reloader" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809232 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="prom-label-proxy" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809238 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b751c1-c3f8-4b3e-aee3-5d3b42cb4d37" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.809422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.809244 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d93c1b-1835-43d7-88cf-371913654014" containerName="kube-rbac-proxy-web" Nov 04 02:07:26.812567 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.812552 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843218 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-web-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843255 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843287 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843351 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843429 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843453 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843480 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843518 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-config-out\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843568 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843595 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843711 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843732 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843755 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843776 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843812 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843854 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843908 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.843954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843941 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.844233 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843959 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.844233 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.843974 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cbwm\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-kube-api-access-9cbwm\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.865543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.865524 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 02:07:26.865641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.865579 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 02:07:26.865821 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.865805 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 02:07:26.918189 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918139 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 02:07:26.918400 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918372 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 02:07:26.918538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918522 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-26iig1f3jpl7f\"" Nov 04 02:07:26.918721 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918703 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:26.918796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918751 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 02:07:26.918858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918802 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 02:07:26.918920 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918895 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 02:07:26.918985 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.918939 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fh89s\"" Nov 04 02:07:26.919081 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.919066 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 02:07:26.919195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.919182 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 02:07:26.919291 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.919270 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 02:07:26.919402 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.919388 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 02:07:26.919476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.919406 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 02:07:26.920534 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.920505 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 02:07:26.945048 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945017 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-config-out\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945048 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945047 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945271 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945063 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945271 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945086 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945271 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945210 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945271 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945268 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945300 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945326 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945396 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945448 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945503 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945529 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945553 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cbwm\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-kube-api-access-9cbwm\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945580 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-web-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945605 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945636 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945663 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945691 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945721 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945747 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.945923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.945863 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.946691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.946616 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.946910 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.946887 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.947029 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.947006 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.947493 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.947468 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.947960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.947935 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5336d54e-01fd-4c53-89ff-29853596b3ad-config-out\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.948827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.948525 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.948827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.948786 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.949049 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.949024 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.949836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.949811 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.950489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.950414 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.950489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.950414 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.950645 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.950516 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.950870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.950713 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.950973 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.950938 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.951368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.951344 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5336d54e-01fd-4c53-89ff-29853596b3ad-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.951695 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.951677 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.951883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.951864 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-web-config\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:26.952303 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:26.952285 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5336d54e-01fd-4c53-89ff-29853596b3ad-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:27.006380 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:27.006359 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cbwm\" (UniqueName: \"kubernetes.io/projected/5336d54e-01fd-4c53-89ff-29853596b3ad-kube-api-access-9cbwm\") pod \"prometheus-k8s-0\" (UID: \"5336d54e-01fd-4c53-89ff-29853596b3ad\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:27.128911 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:27.128877 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:27.557832 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:27.557803 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 02:07:27.560741 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:07:27.560715 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5336d54e_01fd_4c53_89ff_29853596b3ad.slice/crio-84f62146e4cb9cc72e5a480508c1f88bed5fcc796ed8c4c86234b267b9cfe7da WatchSource:0}: Error finding container 84f62146e4cb9cc72e5a480508c1f88bed5fcc796ed8c4c86234b267b9cfe7da: Status 404 returned error can't find the container with id 84f62146e4cb9cc72e5a480508c1f88bed5fcc796ed8c4c86234b267b9cfe7da Nov 04 02:07:27.600550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:27.600510 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"84f62146e4cb9cc72e5a480508c1f88bed5fcc796ed8c4c86234b267b9cfe7da"} Nov 04 02:07:28.425147 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.425113 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83d93c1b-1835-43d7-88cf-371913654014" path="/var/lib/kubelet/pods/83d93c1b-1835-43d7-88cf-371913654014/volumes" Nov 04 02:07:28.555189 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.555151 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c76cc68f9-dcjvf" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerName="console" containerID="cri-o://d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a" gracePeriod=15 Nov 04 02:07:28.558935 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.558908 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-d8cf5d56b-xqwgh" podUID="a50529fc-30c2-4a6f-83db-f90819ad9a94" containerName="console" containerID="cri-o://42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1" gracePeriod=15 Nov 04 02:07:28.605808 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.605778 6073 generic.go:356] "Generic (PLEG): container finished" podID="5336d54e-01fd-4c53-89ff-29853596b3ad" containerID="0177cc6b0e57c304e926a2be200f725de39954454fd19dc864c483288c7538fb" exitCode=0 Nov 04 02:07:28.605943 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.605822 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerDied","Data":"0177cc6b0e57c304e926a2be200f725de39954454fd19dc864c483288c7538fb"} Nov 04 02:07:28.811802 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.811620 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.860790 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.860862 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.860894 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.860971 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.861027 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.861067 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.861099 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") pod \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\" (UID: \"ed3adefd-eb97-468e-912b-fd79b7fa7ee2\") " Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.862078 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.862311 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.862388 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.862991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.862940 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.865499 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.865464 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "kube-api-access-hpmfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:28.865956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.865936 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:28.868251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.868230 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config" pod "ed3adefd-eb97-468e-912b-fd79b7fa7ee2" (UID: "ed3adefd-eb97-468e-912b-fd79b7fa7ee2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:28.903047 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.903023 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:28.962321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962297 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962365 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962385 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962410 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962461 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962496 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.962861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962811 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.962905 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962885 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.962958 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.962938 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.963032 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963008 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") pod \"a50529fc-30c2-4a6f-83db-f90819ad9a94\" (UID: \"a50529fc-30c2-4a6f-83db-f90819ad9a94\") " Nov 04 02:07:28.963581 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963560 6073 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.963734 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963710 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:28.963816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963717 6073 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.963816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963762 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.963816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963766 6073 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-tmp\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.963816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963797 6073 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963817 6073 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963832 6073 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963847 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpmfm\" (UniqueName: \"kubernetes.io/projected/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-kube-api-access-hpmfm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963864 6073 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-console-oauth-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963879 6073 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3adefd-eb97-468e-912b-fd79b7fa7ee2-oauth-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:28.964015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.963921 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:28.966266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.966228 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:28.966409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.966325 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:28.967535 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:28.967506 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z" pod "a50529fc-30c2-4a6f-83db-f90819ad9a94" (UID: "a50529fc-30c2-4a6f-83db-f90819ad9a94"). InnerVolumeSpecName "kube-api-access-hcm2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065865 6073 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-oauth-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065896 6073 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-oauth-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065911 6073 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50529fc-30c2-4a6f-83db-f90819ad9a94-console-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065924 6073 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50529fc-30c2-4a6f-83db-f90819ad9a94-trusted-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065939 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcm2z\" (UniqueName: \"kubernetes.io/projected/a50529fc-30c2-4a6f-83db-f90819ad9a94-kube-api-access-hcm2z\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.066106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.065954 6073 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a50529fc-30c2-4a6f-83db-f90819ad9a94-tmp\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:29.611905 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.611869 6073 generic.go:356] "Generic (PLEG): container finished" podID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerID="d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a" exitCode=0 Nov 04 02:07:29.612361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.611928 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c76cc68f9-dcjvf" Nov 04 02:07:29.612361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.611963 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c76cc68f9-dcjvf" event={"ID":"ed3adefd-eb97-468e-912b-fd79b7fa7ee2","Type":"ContainerDied","Data":"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a"} Nov 04 02:07:29.612361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.612007 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c76cc68f9-dcjvf" event={"ID":"ed3adefd-eb97-468e-912b-fd79b7fa7ee2","Type":"ContainerDied","Data":"1d7ada4fcd5a20050b89ef58b2184bcb26162cd607f84a3590fc6e27429e489a"} Nov 04 02:07:29.612361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.612028 6073 scope.go:122] "RemoveContainer" containerID="d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a" Nov 04 02:07:29.615872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.615843 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"df81927e87ef6d70bedf3961daedff7f820228fc19de3181cf63111fad59f309"} Nov 04 02:07:29.615988 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.615918 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"8708ff2d928ae514f789de9b63148217fb6a49d55c2d0f9fc847c30ca283ff4f"} Nov 04 02:07:29.615988 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.615969 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"a81be4ef4d4afc451346e40ef8cc55d1f540e2dbe526e0941a2a44dc7458108f"} Nov 04 02:07:29.616091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.616011 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"43888db7f4042733cafdaeaf3c45048078b460bfd8a19b8aaf670d25f78b264d"} Nov 04 02:07:29.616091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.616055 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"b5f8c40781fb71e43885bc39f8782f20f2d5ec40304724ca6e7a65a9a82f6a36"} Nov 04 02:07:29.616165 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.616096 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5336d54e-01fd-4c53-89ff-29853596b3ad","Type":"ContainerStarted","Data":"0e67fd4b075f40a0b0df7fe08db4007241f5e1530f45b6b15beff8edffc73b4a"} Nov 04 02:07:29.619177 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.619154 6073 generic.go:356] "Generic (PLEG): container finished" podID="a50529fc-30c2-4a6f-83db-f90819ad9a94" containerID="42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1" exitCode=0 Nov 04 02:07:29.619275 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.619203 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8cf5d56b-xqwgh" Nov 04 02:07:29.619360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.619204 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8cf5d56b-xqwgh" event={"ID":"a50529fc-30c2-4a6f-83db-f90819ad9a94","Type":"ContainerDied","Data":"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1"} Nov 04 02:07:29.619410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.619354 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8cf5d56b-xqwgh" event={"ID":"a50529fc-30c2-4a6f-83db-f90819ad9a94","Type":"ContainerDied","Data":"ea8cd51a79791b88042b44c2145de7a6a860a348a224bf3d4c4b15eb11ff85f1"} Nov 04 02:07:29.620928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.620907 6073 scope.go:122] "RemoveContainer" containerID="d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a" Nov 04 02:07:29.621209 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:29.621190 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a\": container with ID starting with d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a not found: ID does not exist" containerID="d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a" Nov 04 02:07:29.621267 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.621217 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a"} err="failed to get container status \"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a\": rpc error: code = NotFound desc = could not find container \"d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a\": container with ID starting with d2b4e0473b2b1fdb076936b8d66803ffdbd57522807c1170203f834433a0959a not found: ID does not exist" Nov 04 02:07:29.621267 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.621234 6073 scope.go:122] "RemoveContainer" containerID="42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1" Nov 04 02:07:29.628617 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.628602 6073 scope.go:122] "RemoveContainer" containerID="42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1" Nov 04 02:07:29.628835 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:29.628819 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1\": container with ID starting with 42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1 not found: ID does not exist" containerID="42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1" Nov 04 02:07:29.628869 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.628842 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1"} err="failed to get container status \"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1\": rpc error: code = NotFound desc = could not find container \"42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1\": container with ID starting with 42283fb9759c4509e38b2f5a377f40cba41b033c1c4ab29250baec6f5408e1b1 not found: ID does not exist" Nov 04 02:07:29.777035 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.777003 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:07:29.780634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.780609 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c76cc68f9-dcjvf"] Nov 04 02:07:29.889853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.889774 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:07:29.893564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:29.893544 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d8cf5d56b-xqwgh"] Nov 04 02:07:30.024664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:30.024603 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.00959714 podStartE2EDuration="4.024583295s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.986155ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:30.023715439 +0000 UTC m=+27.719405665" watchObservedRunningTime="2026-11-04 02:07:30.024583295 +0000 UTC m=+27.720273493" Nov 04 02:07:30.426950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:30.426907 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a50529fc-30c2-4a6f-83db-f90819ad9a94" path="/var/lib/kubelet/pods/a50529fc-30c2-4a6f-83db-f90819ad9a94/volumes" Nov 04 02:07:30.427773 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:30.427603 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" path="/var/lib/kubelet/pods/ed3adefd-eb97-468e-912b-fd79b7fa7ee2/volumes" Nov 04 02:07:31.629269 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:31.629237 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:31.629718 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:31.629306 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:31.634351 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:31.634312 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:32.129345 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:32.129308 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:07:32.384180 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:32.384109 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g5dnk" Nov 04 02:07:32.643803 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:32.643730 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7956cf9f8-hq7wl" Nov 04 02:07:32.803025 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:32.802998 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-56kwr" Nov 04 02:07:32.811896 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:32.811868 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:07:53.856753 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:53.856712 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:07:57.831598 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:57.831532 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-75dd9456cc-2tsv2" podUID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" containerName="console" containerID="cri-o://10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97" gracePeriod=15 Nov 04 02:07:58.098785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.098763 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:58.229793 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229762 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229804 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229826 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229854 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229896 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229928 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.229976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.229974 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.230307 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.230011 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") pod \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\" (UID: \"01c21249-e6ab-42b6-9391-a9f4ad402b8a\") " Nov 04 02:07:58.230560 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.230401 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:58.230560 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.230486 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:58.230560 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.230522 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:58.230856 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.230635 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:07:58.231069 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.231043 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:07:58.232471 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.232447 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:58.232701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.232683 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:07:58.232781 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.232706 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm" pod "01c21249-e6ab-42b6-9391-a9f4ad402b8a" (UID: "01c21249-e6ab-42b6-9391-a9f4ad402b8a"). InnerVolumeSpecName "kube-api-access-mnpfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:07:58.330872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330839 6073 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-oauth-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.330872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330864 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnpfm\" (UniqueName: \"kubernetes.io/projected/01c21249-e6ab-42b6-9391-a9f4ad402b8a-kube-api-access-mnpfm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.330872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330876 6073 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.330872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330885 6073 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01c21249-e6ab-42b6-9391-a9f4ad402b8a-tmp\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.331135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330894 6073 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-trusted-ca-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.331135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330902 6073 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-console-config\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.331135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330912 6073 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-oauth-serving-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.331135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.330921 6073 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01c21249-e6ab-42b6-9391-a9f4ad402b8a-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:07:58.738325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.738247 6073 generic.go:356] "Generic (PLEG): container finished" podID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" containerID="10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97" exitCode=0 Nov 04 02:07:58.738325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.738293 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75dd9456cc-2tsv2" Nov 04 02:07:58.738524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.738340 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75dd9456cc-2tsv2" event={"ID":"01c21249-e6ab-42b6-9391-a9f4ad402b8a","Type":"ContainerDied","Data":"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97"} Nov 04 02:07:58.738524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.738397 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75dd9456cc-2tsv2" event={"ID":"01c21249-e6ab-42b6-9391-a9f4ad402b8a","Type":"ContainerDied","Data":"8f867fe6cedd34c8f3e89e1588d7a5b1ca8732bc4ae68f1472b06c6f9eac01b1"} Nov 04 02:07:58.738524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.738419 6073 scope.go:122] "RemoveContainer" containerID="10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97" Nov 04 02:07:58.746396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.746376 6073 scope.go:122] "RemoveContainer" containerID="10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97" Nov 04 02:07:58.746644 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:07:58.746628 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97\": container with ID starting with 10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97 not found: ID does not exist" containerID="10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97" Nov 04 02:07:58.746693 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.746652 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97"} err="failed to get container status \"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97\": rpc error: code = NotFound desc = could not find container \"10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97\": container with ID starting with 10710b0d013741620bbacfb5fa7d3d31b4c2ea7e9ebf15653f6f904793d3db97 not found: ID does not exist" Nov 04 02:07:58.901241 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.901211 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:07:58.950895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:07:58.950863 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75dd9456cc-2tsv2"] Nov 04 02:08:00.423915 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:08:00.423883 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" path="/var/lib/kubelet/pods/01c21249-e6ab-42b6-9391-a9f4ad402b8a/volumes" Nov 04 02:08:19.366430 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:08:19.366391 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:08:19.382242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:08:19.382216 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:08:19.830748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:08:19.830720 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 02:08:23.856866 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:08:23.856823 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:08:53.857208 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:08:53.857168 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:09:23.857764 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:09:23.857727 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:09:26.662776 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.662745 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt"] Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663051 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a50529fc-30c2-4a6f-83db-f90819ad9a94" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663061 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a50529fc-30c2-4a6f-83db-f90819ad9a94" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663071 6073 cpu_manager.go:414] "containerMap: removing container" podUID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663075 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663083 6073 cpu_manager.go:414] "containerMap: removing container" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663100 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663159 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="a50529fc-30c2-4a6f-83db-f90819ad9a94" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663167 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="01c21249-e6ab-42b6-9391-a9f4ad402b8a" containerName="console" Nov 04 02:09:26.663244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.663173 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed3adefd-eb97-468e-912b-fd79b7fa7ee2" containerName="console" Nov 04 02:09:26.665012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.664997 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.718618 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.718596 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 02:09:26.768254 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.768221 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 02:09:26.768422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.768296 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-lhfcf\"" Nov 04 02:09:26.768987 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.768970 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt"] Nov 04 02:09:26.769377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.769359 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl"] Nov 04 02:09:26.771420 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.771408 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.775543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.775521 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.775658 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.775559 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.775712 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.775696 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8p59\" (UniqueName: \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.823378 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.823353 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-64nfq\"" Nov 04 02:09:26.839176 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.839153 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl"] Nov 04 02:09:26.876953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.876924 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-585fx\" (UniqueName: \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.877087 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.876961 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8p59\" (UniqueName: \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.877087 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.876987 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.877171 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.877121 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.877171 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.877145 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.877248 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.877183 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.877502 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.877483 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.877544 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.877507 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.900629 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.900573 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x"] Nov 04 02:09:26.903157 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.903139 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9"] Nov 04 02:09:26.903282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.903263 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:26.905047 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.905030 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:26.937801 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.937780 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8p59\" (UniqueName: \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.958905 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.958883 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-z79vp\"" Nov 04 02:09:26.959026 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.958947 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-hv2kt\"" Nov 04 02:09:26.961205 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.961188 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9"] Nov 04 02:09:26.964026 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.964007 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x"] Nov 04 02:09:26.967636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.967612 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l"] Nov 04 02:09:26.969890 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.969875 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:26.977270 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.977250 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:09:26.978222 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978205 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:26.978300 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978242 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-585fx\" (UniqueName: \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.978376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978301 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85r5s\" (UniqueName: \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:26.978376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978325 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7rzw\" (UniqueName: \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:26.978490 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978390 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.978659 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978636 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:26.978731 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978715 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:26.978790 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978750 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.978853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978787 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:26.978853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.978803 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:26.979080 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:26.979060 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:27.022144 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.022113 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-lsqxd\"" Nov 04 02:09:27.034437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.034044 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l"] Nov 04 02:09:27.044878 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.044849 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-585fx\" (UniqueName: \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:27.079994 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.079961 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7rzw\" (UniqueName: \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.080148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080010 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59685\" (UniqueName: \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.080148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080032 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.080148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080052 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.080324 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080180 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.080324 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080231 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.080324 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080286 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.080520 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080362 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85r5s\" (UniqueName: \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.080520 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080400 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.080588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080516 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.080633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080573 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.080633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080600 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.080704 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.080641 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.082898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.082885 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:09:27.143128 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.143099 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85r5s\" (UniqueName: \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.143274 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.143253 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7rzw\" (UniqueName: \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.181020 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.180944 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.181020 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.181000 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59685\" (UniqueName: \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.181244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.181043 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.181314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.181295 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.181439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.181421 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.215787 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.215750 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:09:27.221240 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.221200 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:09:27.242185 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.242161 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59685\" (UniqueName: \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.285304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.285089 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:09:27.348879 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.348853 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt"] Nov 04 02:09:27.352448 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:27.352413 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdde932_def9_4ff2_bf1a_ace0f4698008.slice/crio-4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3 WatchSource:0}: Error finding container 4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3: Status 404 returned error can't find the container with id 4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3 Nov 04 02:09:27.354661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.354644 6073 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:09:27.450286 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.450221 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl"] Nov 04 02:09:27.453374 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:27.453343 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda389607b_92e4_41d4_8145_3ca6d8b328a4.slice/crio-a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839 WatchSource:0}: Error finding container a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839: Status 404 returned error can't find the container with id a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839 Nov 04 02:09:27.598348 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.598299 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x"] Nov 04 02:09:27.600927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.600899 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9"] Nov 04 02:09:27.601406 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:27.601384 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb73e72_2fa2_4c98_bb5a_452923450590.slice/crio-64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb WatchSource:0}: Error finding container 64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb: Status 404 returned error can't find the container with id 64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb Nov 04 02:09:27.606090 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:27.606067 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01eb26c_f9c5_44b0_91a0_fe558a128257.slice/crio-ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8 WatchSource:0}: Error finding container ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8: Status 404 returned error can't find the container with id ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8 Nov 04 02:09:27.656440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:27.656401 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l"] Nov 04 02:09:27.659286 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:27.659251 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f74df6d_1a62_4219_87ff_959c52546c75.slice/crio-1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40 WatchSource:0}: Error finding container 1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40: Status 404 returned error can't find the container with id 1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40 Nov 04 02:09:28.007604 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.007572 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z"] Nov 04 02:09:28.010478 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.010460 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.062494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.062465 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-tfmkr\"" Nov 04 02:09:28.071480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.069883 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z"] Nov 04 02:09:28.076224 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.076195 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerStarted","Data":"64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb"} Nov 04 02:09:28.077718 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.077694 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" event={"ID":"a389607b-92e4-41d4-8145-3ca6d8b328a4","Type":"ContainerStarted","Data":"a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839"} Nov 04 02:09:28.079233 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.079203 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" event={"ID":"d01eb26c-f9c5-44b0-91a0-fe558a128257","Type":"ContainerStarted","Data":"ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8"} Nov 04 02:09:28.081066 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.081039 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" event={"ID":"4cdde932-def9-4ff2-bf1a-ace0f4698008","Type":"ContainerStarted","Data":"4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3"} Nov 04 02:09:28.085440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.085412 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" event={"ID":"0f74df6d-1a62-4219-87ff-959c52546c75","Type":"ContainerStarted","Data":"1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40"} Nov 04 02:09:28.088976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.088951 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.089065 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.089047 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drxxm\" (UniqueName: \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.089133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.089072 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.190253 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.190204 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drxxm\" (UniqueName: \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.190471 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.190265 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.190471 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.190368 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.190793 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.190766 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.190916 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.190779 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.251731 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.251694 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drxxm\" (UniqueName: \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.323572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.323537 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:09:28.712551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:28.712463 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z"] Nov 04 02:09:28.729838 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:28.727499 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df1e2bb_ac92_4663_b21b_48b0d61bc17f.slice/crio-c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e WatchSource:0}: Error finding container c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e: Status 404 returned error can't find the container with id c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e Nov 04 02:09:29.094851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:29.094761 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerStarted","Data":"c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e"} Nov 04 02:09:30.971368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:30.970317 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc"] Nov 04 02:09:30.974636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:30.973930 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.076944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.075590 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-sgmgc\"" Nov 04 02:09:31.076944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.076863 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc"] Nov 04 02:09:31.119584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.119268 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjpz7\" (UniqueName: \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.119584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.119350 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.119584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.119426 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.221779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.220756 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.221779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.220888 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjpz7\" (UniqueName: \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.221779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.220940 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.221779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.221416 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.221779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.221678 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.296364 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.296264 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjpz7\" (UniqueName: \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:31.594058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:31.593806 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:09:32.023659 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:32.023624 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc"] Nov 04 02:09:32.029731 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:09:32.029701 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e00307_c78b_4a5f_9852_089443996c1a.slice/crio-f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e WatchSource:0}: Error finding container f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e: Status 404 returned error can't find the container with id f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e Nov 04 02:09:32.119740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:32.119686 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerStarted","Data":"f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e"} Nov 04 02:09:33.881389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.879990 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j"] Nov 04 02:09:33.885252 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.884212 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:33.972903 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.972844 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 02:09:33.973071 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.972968 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 02:09:33.973071 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.973021 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-c7r69\"" Nov 04 02:09:33.986118 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.986052 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 02:09:33.987853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:33.987826 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j"] Nov 04 02:09:34.055999 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.055956 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056046 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056080 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056193 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056233 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056250 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056265 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056324 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.056536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.056404 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gsdx\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-kube-api-access-4gsdx\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.157998 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.157920 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.157998 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.157974 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158004 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158045 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158072 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158111 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158154 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158263 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.158416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158307 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gsdx\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-kube-api-access-4gsdx\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.159011 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158771 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.159011 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.158884 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.159199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.159022 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.159521 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.159431 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.160231 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.160205 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.170911 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.170834 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.187080 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.227912 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.227884 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.234010 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.233955 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gsdx\" (UniqueName: \"kubernetes.io/projected/1e1dfd17-3ead-4067-b9b3-79fe50be2195-kube-api-access-4gsdx\") pod \"openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j\" (UID: \"1e1dfd17-3ead-4067-b9b3-79fe50be2195\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.502233 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.501868 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:34.592117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.589776 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb"] Nov 04 02:09:34.593540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.593516 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.654637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.654603 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb"] Nov 04 02:09:34.764891 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.764803 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4gsw\" (UniqueName: \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.765058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.764898 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.765058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.765007 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.865817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.865780 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.865997 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.865908 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.865997 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.865961 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4gsw\" (UniqueName: \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.866700 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.866678 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.866952 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.866932 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:34.930672 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:34.930534 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4gsw\" (UniqueName: \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:35.024230 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:35.024074 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j"] Nov 04 02:09:35.145743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:35.145640 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" event={"ID":"1e1dfd17-3ead-4067-b9b3-79fe50be2195","Type":"ContainerStarted","Data":"811bde6bdcb6745e71eab68ca72d29759c23e9d3131b5b0bcf3b9dd6a4e9ba48"} Nov 04 02:09:35.218373 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:35.218070 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:09:35.668416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:35.663483 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb"] Nov 04 02:09:36.155576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:36.155525 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" event={"ID":"c5415a7a-eefc-48ae-885d-eb323d150cd1","Type":"ContainerStarted","Data":"704df69a0c34c09865db0a909680571847439663d4b6b7ebc99ff2a93df53cfa"} Nov 04 02:09:43.261090 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:43.261056 6073 generic.go:356] "Generic (PLEG): container finished" podID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerID="ed80ccdeabcc856425b1acb2817b4215165c62fffb9d46a63b0b2f2b16bcc7b9" exitCode=0 Nov 04 02:09:43.261553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:43.261164 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerDied","Data":"ed80ccdeabcc856425b1acb2817b4215165c62fffb9d46a63b0b2f2b16bcc7b9"} Nov 04 02:09:43.267562 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:43.267534 6073 generic.go:356] "Generic (PLEG): container finished" podID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerID="c6d91b56ea381dd3ebdb091eeda1fcb10182c6cb6da124266c7f2166c20fb81e" exitCode=0 Nov 04 02:09:43.267664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:43.267596 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" event={"ID":"a389607b-92e4-41d4-8145-3ca6d8b328a4","Type":"ContainerDied","Data":"c6d91b56ea381dd3ebdb091eeda1fcb10182c6cb6da124266c7f2166c20fb81e"} Nov 04 02:09:44.274964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.274928 6073 generic.go:356] "Generic (PLEG): container finished" podID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerID="a4af386d7db652039fe5a116b670d0acb07fe6bdb7fdf5048f2768eacfc69b50" exitCode=0 Nov 04 02:09:44.275376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.275025 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" event={"ID":"d01eb26c-f9c5-44b0-91a0-fe558a128257","Type":"ContainerDied","Data":"a4af386d7db652039fe5a116b670d0acb07fe6bdb7fdf5048f2768eacfc69b50"} Nov 04 02:09:44.277253 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.277225 6073 generic.go:356] "Generic (PLEG): container finished" podID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerID="167cdd48868c60643c321b611872d6b85927a0a1ee42c30e12d98a4cda7a6d7a" exitCode=0 Nov 04 02:09:44.277377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.277309 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" event={"ID":"4cdde932-def9-4ff2-bf1a-ace0f4698008","Type":"ContainerDied","Data":"167cdd48868c60643c321b611872d6b85927a0a1ee42c30e12d98a4cda7a6d7a"} Nov 04 02:09:44.278991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.278969 6073 generic.go:356] "Generic (PLEG): container finished" podID="32e00307-c78b-4a5f-9852-089443996c1a" containerID="c72370542bcb5c39e60e448770b96d989be40f9dc2eff5c2576615a68a969610" exitCode=0 Nov 04 02:09:44.279081 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.279002 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerDied","Data":"c72370542bcb5c39e60e448770b96d989be40f9dc2eff5c2576615a68a969610"} Nov 04 02:09:44.388036 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.388008 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:09:44.388146 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.388090 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:09:44.388146 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:44.388132 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:09:45.286082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.286043 6073 generic.go:356] "Generic (PLEG): container finished" podID="32e00307-c78b-4a5f-9852-089443996c1a" containerID="7b294e424a27a30f76ac007cd3a11924e039310655e3377f125f7bd9956375e7" exitCode=0 Nov 04 02:09:45.286543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.286140 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerDied","Data":"7b294e424a27a30f76ac007cd3a11924e039310655e3377f125f7bd9956375e7"} Nov 04 02:09:45.288165 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.288042 6073 generic.go:356] "Generic (PLEG): container finished" podID="0f74df6d-1a62-4219-87ff-959c52546c75" containerID="e755ad596fe6530a54cf62cec2972b4735ec41c036d2df2ce1471a1319c8c6f9" exitCode=0 Nov 04 02:09:45.288165 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.288130 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" event={"ID":"0f74df6d-1a62-4219-87ff-959c52546c75","Type":"ContainerDied","Data":"e755ad596fe6530a54cf62cec2972b4735ec41c036d2df2ce1471a1319c8c6f9"} Nov 04 02:09:45.290110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.290086 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" event={"ID":"1e1dfd17-3ead-4067-b9b3-79fe50be2195","Type":"ContainerStarted","Data":"ce2435999d44e4110d3402c00db8886b1a364f4d351322e12697127a2c16ef36"} Nov 04 02:09:45.291660 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.291635 6073 generic.go:356] "Generic (PLEG): container finished" podID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerID="6aef704905dd00f2f2e08c8f54e5c1f2e3515ee2521610038f660bc4bdb6d71c" exitCode=0 Nov 04 02:09:45.291769 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.291717 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerDied","Data":"6aef704905dd00f2f2e08c8f54e5c1f2e3515ee2521610038f660bc4bdb6d71c"} Nov 04 02:09:45.293624 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.293600 6073 generic.go:356] "Generic (PLEG): container finished" podID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerID="bfc5c44279b05e0990017e237be5affd2f9dac8ee1afcf5692b224589a8e97bd" exitCode=0 Nov 04 02:09:45.293711 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.293633 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" event={"ID":"c5415a7a-eefc-48ae-885d-eb323d150cd1","Type":"ContainerDied","Data":"bfc5c44279b05e0990017e237be5affd2f9dac8ee1afcf5692b224589a8e97bd"} Nov 04 02:09:45.502699 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.502586 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:45.507939 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.507911 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:45.523888 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:45.523829 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" podStartSLOduration=3.170430963 podStartE2EDuration="12.523809667s" totalImagesPullingTime="9.353378704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:45.52338878 +0000 UTC m=+163.219078982" watchObservedRunningTime="2026-11-04 02:09:45.523809667 +0000 UTC m=+163.219499872" Nov 04 02:09:46.305510 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.305475 6073 generic.go:356] "Generic (PLEG): container finished" podID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerID="3eeb08dc5523f674e4d1611f7d2448e724c31640cc292d2a0b9be217205bf291" exitCode=0 Nov 04 02:09:46.306119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.305577 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" event={"ID":"a389607b-92e4-41d4-8145-3ca6d8b328a4","Type":"ContainerDied","Data":"3eeb08dc5523f674e4d1611f7d2448e724c31640cc292d2a0b9be217205bf291"} Nov 04 02:09:46.308225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.308197 6073 generic.go:356] "Generic (PLEG): container finished" podID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerID="d1b0f32b7bf40cd6062fb1d7799c6e247f6b1c765915a1e2f1950f5d84375aa7" exitCode=0 Nov 04 02:09:46.309660 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.309635 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" event={"ID":"4cdde932-def9-4ff2-bf1a-ace0f4698008","Type":"ContainerDied","Data":"d1b0f32b7bf40cd6062fb1d7799c6e247f6b1c765915a1e2f1950f5d84375aa7"} Nov 04 02:09:46.309807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.309688 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:46.310791 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:46.310772 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j" Nov 04 02:09:47.318909 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.318871 6073 generic.go:356] "Generic (PLEG): container finished" podID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerID="547031efdc2a365a1d861bea84c479292370eb65845faf05a7b2237bdf812c44" exitCode=0 Nov 04 02:09:47.319449 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.318970 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" event={"ID":"c5415a7a-eefc-48ae-885d-eb323d150cd1","Type":"ContainerDied","Data":"547031efdc2a365a1d861bea84c479292370eb65845faf05a7b2237bdf812c44"} Nov 04 02:09:47.323578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.323549 6073 generic.go:356] "Generic (PLEG): container finished" podID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerID="0fa1ff03480753e3798ce1ae21d72f2b474fb1130176d7ba38a4ea8f4ea2a37d" exitCode=0 Nov 04 02:09:47.323696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.323667 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerDied","Data":"0fa1ff03480753e3798ce1ae21d72f2b474fb1130176d7ba38a4ea8f4ea2a37d"} Nov 04 02:09:47.327848 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.327820 6073 generic.go:356] "Generic (PLEG): container finished" podID="0f74df6d-1a62-4219-87ff-959c52546c75" containerID="ff601122515011533ab476898f62424bb3822d6a1dda1ef72090d90a9e481211" exitCode=0 Nov 04 02:09:47.328960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:47.328937 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" event={"ID":"0f74df6d-1a62-4219-87ff-959c52546c75","Type":"ContainerDied","Data":"ff601122515011533ab476898f62424bb3822d6a1dda1ef72090d90a9e481211"} Nov 04 02:09:50.354882 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:50.353994 6073 generic.go:356] "Generic (PLEG): container finished" podID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerID="9d7e06f6d061f2cbc70af34999bb513cae183f3e16d3bae84c788cb04de94dd2" exitCode=0 Nov 04 02:09:50.354882 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:09:50.354053 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerDied","Data":"9d7e06f6d061f2cbc70af34999bb513cae183f3e16d3bae84c788cb04de94dd2"} Nov 04 02:09:53.908270 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:09:53.908188 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:10:00.439476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.439387 6073 generic.go:356] "Generic (PLEG): container finished" podID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerID="939034d1bdd852b58ed5147df6310dd70bd478888a44eecebcc9d3a94117406f" exitCode=0 Nov 04 02:10:00.439993 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.439512 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" event={"ID":"d01eb26c-f9c5-44b0-91a0-fe558a128257","Type":"ContainerDied","Data":"939034d1bdd852b58ed5147df6310dd70bd478888a44eecebcc9d3a94117406f"} Nov 04 02:10:00.448948 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.447292 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerStarted","Data":"80fa2866de19a2c31d25fa551ba8ad65d7f87a785fbd6fe1a130dff3f1f4fe1f"} Nov 04 02:10:00.462680 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.462633 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerStarted","Data":"573f5242c0c0dbb25a96688df4e59b31f6b4b6a09cbf2cf7cc92c3d1183fff1e"} Nov 04 02:10:00.684819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.684589 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" podStartSLOduration=3.472713062 podStartE2EDuration="30.68456637s" totalImagesPullingTime="27.211853308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:00.682250202 +0000 UTC m=+178.377940402" watchObservedRunningTime="2026-11-04 02:10:00.68456637 +0000 UTC m=+178.380256575" Nov 04 02:10:00.801827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:00.801161 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" podStartSLOduration=3.223843846 podStartE2EDuration="33.801150951s" totalImagesPullingTime="30.577307105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:00.800505128 +0000 UTC m=+178.496195329" watchObservedRunningTime="2026-11-04 02:10:00.801150951 +0000 UTC m=+178.496841147" Nov 04 02:10:01.472055 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.472020 6073 generic.go:356] "Generic (PLEG): container finished" podID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerID="97fb3b54f1c18ceb710b3ee6bb25aa79ca96bf696054958eff3fd6cd8da084d9" exitCode=0 Nov 04 02:10:01.472444 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.472103 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" event={"ID":"d01eb26c-f9c5-44b0-91a0-fe558a128257","Type":"ContainerDied","Data":"97fb3b54f1c18ceb710b3ee6bb25aa79ca96bf696054958eff3fd6cd8da084d9"} Nov 04 02:10:01.475894 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.475862 6073 generic.go:356] "Generic (PLEG): container finished" podID="32e00307-c78b-4a5f-9852-089443996c1a" containerID="80fa2866de19a2c31d25fa551ba8ad65d7f87a785fbd6fe1a130dff3f1f4fe1f" exitCode=0 Nov 04 02:10:01.476039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.475944 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerDied","Data":"80fa2866de19a2c31d25fa551ba8ad65d7f87a785fbd6fe1a130dff3f1f4fe1f"} Nov 04 02:10:01.479098 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.478867 6073 generic.go:356] "Generic (PLEG): container finished" podID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerID="573f5242c0c0dbb25a96688df4e59b31f6b4b6a09cbf2cf7cc92c3d1183fff1e" exitCode=0 Nov 04 02:10:01.479098 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:01.478948 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerDied","Data":"573f5242c0c0dbb25a96688df4e59b31f6b4b6a09cbf2cf7cc92c3d1183fff1e"} Nov 04 02:10:02.484611 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.484579 6073 generic.go:356] "Generic (PLEG): container finished" podID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerID="e2c0fd4ffcc6deced8d6acf0a84ea59a0fa8dd1f1f1d882d8a93d8578732e79e" exitCode=0 Nov 04 02:10:02.485020 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.484666 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" event={"ID":"4cdde932-def9-4ff2-bf1a-ace0f4698008","Type":"ContainerDied","Data":"e2c0fd4ffcc6deced8d6acf0a84ea59a0fa8dd1f1f1d882d8a93d8578732e79e"} Nov 04 02:10:02.486704 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.486684 6073 generic.go:356] "Generic (PLEG): container finished" podID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerID="fff2fcba876d6a404112bfe41ea7cc761a69507e17c2cfd7bd9517eb633623be" exitCode=0 Nov 04 02:10:02.486819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.486779 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" event={"ID":"a389607b-92e4-41d4-8145-3ca6d8b328a4","Type":"ContainerDied","Data":"fff2fcba876d6a404112bfe41ea7cc761a69507e17c2cfd7bd9517eb633623be"} Nov 04 02:10:02.637882 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.637855 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:10:02.669092 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.669067 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:10:02.693270 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.693250 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:10:02.791633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791608 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\") pod \"d01eb26c-f9c5-44b0-91a0-fe558a128257\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " Nov 04 02:10:02.791767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791649 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\" (UniqueName: \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\") pod \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " Nov 04 02:10:02.791767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791697 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\") pod \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " Nov 04 02:10:02.791767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791725 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\") pod \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\" (UID: \"7df1e2bb-ac92-4663-b21b-48b0d61bc17f\") " Nov 04 02:10:02.791895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791784 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\") pod \"d01eb26c-f9c5-44b0-91a0-fe558a128257\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " Nov 04 02:10:02.791895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791831 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\" (UniqueName: \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\") pod \"32e00307-c78b-4a5f-9852-089443996c1a\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " Nov 04 02:10:02.791895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791867 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\") pod \"32e00307-c78b-4a5f-9852-089443996c1a\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " Nov 04 02:10:02.792045 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.791931 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\" (UniqueName: \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\") pod \"d01eb26c-f9c5-44b0-91a0-fe558a128257\" (UID: \"d01eb26c-f9c5-44b0-91a0-fe558a128257\") " Nov 04 02:10:02.792101 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.792089 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\") pod \"32e00307-c78b-4a5f-9852-089443996c1a\" (UID: \"32e00307-c78b-4a5f-9852-089443996c1a\") " Nov 04 02:10:02.792713 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.792669 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle" pod "d01eb26c-f9c5-44b0-91a0-fe558a128257" (UID: "d01eb26c-f9c5-44b0-91a0-fe558a128257"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.793145 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.793123 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle" pod "7df1e2bb-ac92-4663-b21b-48b0d61bc17f" (UID: "7df1e2bb-ac92-4663-b21b-48b0d61bc17f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.793272 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.793167 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle" pod "32e00307-c78b-4a5f-9852-089443996c1a" (UID: "32e00307-c78b-4a5f-9852-089443996c1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.795094 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.795062 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm" pod "7df1e2bb-ac92-4663-b21b-48b0d61bc17f" (UID: "7df1e2bb-ac92-4663-b21b-48b0d61bc17f"). InnerVolumeSpecName "kube-api-access-drxxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:02.795188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.795160 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7" pod "32e00307-c78b-4a5f-9852-089443996c1a" (UID: "32e00307-c78b-4a5f-9852-089443996c1a"). InnerVolumeSpecName "kube-api-access-mjpz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:02.795296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.795279 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s" pod "d01eb26c-f9c5-44b0-91a0-fe558a128257" (UID: "d01eb26c-f9c5-44b0-91a0-fe558a128257"). InnerVolumeSpecName "kube-api-access-85r5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:02.798130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.798098 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util" pod "d01eb26c-f9c5-44b0-91a0-fe558a128257" (UID: "d01eb26c-f9c5-44b0-91a0-fe558a128257"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.798693 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.798667 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util" pod "7df1e2bb-ac92-4663-b21b-48b0d61bc17f" (UID: "7df1e2bb-ac92-4663-b21b-48b0d61bc17f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.800505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.800481 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util" pod "32e00307-c78b-4a5f-9852-089443996c1a" (UID: "32e00307-c78b-4a5f-9852-089443996c1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893394 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893422 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893432 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893441 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjpz7\" (UniqueName: \"kubernetes.io/projected/32e00307-c78b-4a5f-9852-089443996c1a-kube-api-access-mjpz7\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893450 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893459 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85r5s\" (UniqueName: \"kubernetes.io/projected/d01eb26c-f9c5-44b0-91a0-fe558a128257-kube-api-access-85r5s\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893467 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32e00307-c78b-4a5f-9852-089443996c1a-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893475 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d01eb26c-f9c5-44b0-91a0-fe558a128257-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:02.893482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:02.893482 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drxxm\" (UniqueName: \"kubernetes.io/projected/7df1e2bb-ac92-4663-b21b-48b0d61bc17f-kube-api-access-drxxm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.082063 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:10:03.082023 6073 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f74df6d_1a62_4219_87ff_959c52546c75.slice/crio-f817547612774b38d656740a3e0306c3c5e764ff1ef27e34df844a8807048b87.scope\": RecentStats: unable to find data in memory cache]" Nov 04 02:10:03.493480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.493388 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" event={"ID":"7df1e2bb-ac92-4663-b21b-48b0d61bc17f","Type":"ContainerDied","Data":"c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e"} Nov 04 02:10:03.493480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.493446 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c158db85866c9e57e96cac125eabbd1c644486d12e9528527e20b9c5e4861f5e" Nov 04 02:10:03.493480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.493409 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447ks5z" Nov 04 02:10:03.495540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.495511 6073 generic.go:356] "Generic (PLEG): container finished" podID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerID="9b60626d707d8058ca4ae2862d49f6adb98a852193e6589e0f44e386ca39de4d" exitCode=0 Nov 04 02:10:03.495667 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.495584 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" event={"ID":"c5415a7a-eefc-48ae-885d-eb323d150cd1","Type":"ContainerDied","Data":"9b60626d707d8058ca4ae2862d49f6adb98a852193e6589e0f44e386ca39de4d"} Nov 04 02:10:03.497811 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.497782 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerStarted","Data":"e1e59d08b5ec51ee7b9a166c91e4f8e8804d55b20e2c2923fbd884103498a12d"} Nov 04 02:10:03.499630 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.499614 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" Nov 04 02:10:03.499742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.499615 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c98dq9" event={"ID":"d01eb26c-f9c5-44b0-91a0-fe558a128257","Type":"ContainerDied","Data":"ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8"} Nov 04 02:10:03.499742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.499727 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba0f7fc2e889e48b00ea15b447ec7b12e148a28c0e504d8f10287af74eaf45e8" Nov 04 02:10:03.501614 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.501589 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" event={"ID":"32e00307-c78b-4a5f-9852-089443996c1a","Type":"ContainerDied","Data":"f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e"} Nov 04 02:10:03.501718 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.501635 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b7d608f937c46cbcb5b2daf85d1f7763fe5c17ac5bd1a7518194849d33fc7e" Nov 04 02:10:03.501718 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.501597 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715q6rc" Nov 04 02:10:03.512540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.512514 6073 generic.go:356] "Generic (PLEG): container finished" podID="0f74df6d-1a62-4219-87ff-959c52546c75" containerID="f817547612774b38d656740a3e0306c3c5e764ff1ef27e34df844a8807048b87" exitCode=0 Nov 04 02:10:03.512642 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.512600 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" event={"ID":"0f74df6d-1a62-4219-87ff-959c52546c75","Type":"ContainerDied","Data":"f817547612774b38d656740a3e0306c3c5e764ff1ef27e34df844a8807048b87"} Nov 04 02:10:03.656777 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.656752 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:10:03.676852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.676829 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:10:03.799870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.799836 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\") pod \"4cdde932-def9-4ff2-bf1a-ace0f4698008\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " Nov 04 02:10:03.800054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.799884 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\") pod \"a389607b-92e4-41d4-8145-3ca6d8b328a4\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " Nov 04 02:10:03.800054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.799922 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\") pod \"a389607b-92e4-41d4-8145-3ca6d8b328a4\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " Nov 04 02:10:03.800054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.799966 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\" (UniqueName: \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\") pod \"4cdde932-def9-4ff2-bf1a-ace0f4698008\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " Nov 04 02:10:03.800054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.800004 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\" (UniqueName: \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\") pod \"a389607b-92e4-41d4-8145-3ca6d8b328a4\" (UID: \"a389607b-92e4-41d4-8145-3ca6d8b328a4\") " Nov 04 02:10:03.800054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.800037 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\") pod \"4cdde932-def9-4ff2-bf1a-ace0f4698008\" (UID: \"4cdde932-def9-4ff2-bf1a-ace0f4698008\") " Nov 04 02:10:03.800655 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.800627 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle" pod "4cdde932-def9-4ff2-bf1a-ace0f4698008" (UID: "4cdde932-def9-4ff2-bf1a-ace0f4698008"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:03.800784 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.800672 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle" pod "a389607b-92e4-41d4-8145-3ca6d8b328a4" (UID: "a389607b-92e4-41d4-8145-3ca6d8b328a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:03.802820 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.802793 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx" pod "a389607b-92e4-41d4-8145-3ca6d8b328a4" (UID: "a389607b-92e4-41d4-8145-3ca6d8b328a4"). InnerVolumeSpecName "kube-api-access-585fx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:03.802922 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.802819 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59" pod "4cdde932-def9-4ff2-bf1a-ace0f4698008" (UID: "4cdde932-def9-4ff2-bf1a-ace0f4698008"). InnerVolumeSpecName "kube-api-access-z8p59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:03.805794 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.805774 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util" pod "a389607b-92e4-41d4-8145-3ca6d8b328a4" (UID: "a389607b-92e4-41d4-8145-3ca6d8b328a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:03.805849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.805794 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util" pod "4cdde932-def9-4ff2-bf1a-ace0f4698008" (UID: "4cdde932-def9-4ff2-bf1a-ace0f4698008"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:03.901132 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901106 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.901132 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901133 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.901281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901142 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a389607b-92e4-41d4-8145-3ca6d8b328a4-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.901281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901151 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8p59\" (UniqueName: \"kubernetes.io/projected/4cdde932-def9-4ff2-bf1a-ace0f4698008-kube-api-access-z8p59\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.901281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901161 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-585fx\" (UniqueName: \"kubernetes.io/projected/a389607b-92e4-41d4-8145-3ca6d8b328a4-kube-api-access-585fx\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.901281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.901171 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cdde932-def9-4ff2-bf1a-ace0f4698008-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:03.914920 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:03.914874 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" podStartSLOduration=3.641279797 podStartE2EDuration="37.914859804s" totalImagesPullingTime="34.273580007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:03.91401104 +0000 UTC m=+181.609701240" watchObservedRunningTime="2026-11-04 02:10:03.914859804 +0000 UTC m=+181.610550004" Nov 04 02:10:04.518735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.518703 6073 generic.go:356] "Generic (PLEG): container finished" podID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerID="e1e59d08b5ec51ee7b9a166c91e4f8e8804d55b20e2c2923fbd884103498a12d" exitCode=0 Nov 04 02:10:04.519179 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.518799 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerDied","Data":"e1e59d08b5ec51ee7b9a166c91e4f8e8804d55b20e2c2923fbd884103498a12d"} Nov 04 02:10:04.520540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.520522 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" Nov 04 02:10:04.520639 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.520521 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847artmgl" event={"ID":"a389607b-92e4-41d4-8145-3ca6d8b328a4","Type":"ContainerDied","Data":"a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839"} Nov 04 02:10:04.520710 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.520636 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a11f9b5ca300967e7565506646dc85ac274cf309b2deb5d252abc7f09f71d839" Nov 04 02:10:04.522589 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.522450 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" event={"ID":"4cdde932-def9-4ff2-bf1a-ace0f4698008","Type":"ContainerDied","Data":"4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3"} Nov 04 02:10:04.522589 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.522486 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8c27c56c131de177b7eb09632f5761e21427ed86494d466fc36722342720c3" Nov 04 02:10:04.522589 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.522459 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6bflvt" Nov 04 02:10:04.673225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.673200 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:10:04.676197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.676175 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:10:04.809437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809409 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\" (UniqueName: \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\") pod \"c5415a7a-eefc-48ae-885d-eb323d150cd1\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " Nov 04 02:10:04.809620 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809491 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\" (UniqueName: \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\") pod \"0f74df6d-1a62-4219-87ff-959c52546c75\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " Nov 04 02:10:04.809620 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809521 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\") pod \"0f74df6d-1a62-4219-87ff-959c52546c75\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " Nov 04 02:10:04.809620 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809565 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\") pod \"c5415a7a-eefc-48ae-885d-eb323d150cd1\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " Nov 04 02:10:04.809620 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809591 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\") pod \"c5415a7a-eefc-48ae-885d-eb323d150cd1\" (UID: \"c5415a7a-eefc-48ae-885d-eb323d150cd1\") " Nov 04 02:10:04.809844 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.809652 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\") pod \"0f74df6d-1a62-4219-87ff-959c52546c75\" (UID: \"0f74df6d-1a62-4219-87ff-959c52546c75\") " Nov 04 02:10:04.810214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.810175 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle" pod "c5415a7a-eefc-48ae-885d-eb323d150cd1" (UID: "c5415a7a-eefc-48ae-885d-eb323d150cd1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:04.810326 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.810236 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle" pod "0f74df6d-1a62-4219-87ff-959c52546c75" (UID: "0f74df6d-1a62-4219-87ff-959c52546c75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:04.810517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.810483 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:04.810517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.810514 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:04.811814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.811793 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw" pod "c5415a7a-eefc-48ae-885d-eb323d150cd1" (UID: "c5415a7a-eefc-48ae-885d-eb323d150cd1"). InnerVolumeSpecName "kube-api-access-z4gsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:04.812201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.812185 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685" pod "0f74df6d-1a62-4219-87ff-959c52546c75" (UID: "0f74df6d-1a62-4219-87ff-959c52546c75"). InnerVolumeSpecName "kube-api-access-59685". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:04.815893 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.815871 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util" pod "c5415a7a-eefc-48ae-885d-eb323d150cd1" (UID: "c5415a7a-eefc-48ae-885d-eb323d150cd1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:04.816171 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.816094 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util" pod "0f74df6d-1a62-4219-87ff-959c52546c75" (UID: "0f74df6d-1a62-4219-87ff-959c52546c75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:04.911839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.911802 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4gsw\" (UniqueName: \"kubernetes.io/projected/c5415a7a-eefc-48ae-885d-eb323d150cd1-kube-api-access-z4gsw\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:04.911839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.911829 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59685\" (UniqueName: \"kubernetes.io/projected/0f74df6d-1a62-4219-87ff-959c52546c75-kube-api-access-59685\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:04.911839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.911838 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f74df6d-1a62-4219-87ff-959c52546c75-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:04.911839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:04.911847 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5415a7a-eefc-48ae-885d-eb323d150cd1-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:05.528550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.528519 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" Nov 04 02:10:05.528550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.528543 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqrgxb" event={"ID":"c5415a7a-eefc-48ae-885d-eb323d150cd1","Type":"ContainerDied","Data":"704df69a0c34c09865db0a909680571847439663d4b6b7ebc99ff2a93df53cfa"} Nov 04 02:10:05.529149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.528590 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="704df69a0c34c09865db0a909680571847439663d4b6b7ebc99ff2a93df53cfa" Nov 04 02:10:05.530447 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.530422 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" event={"ID":"0f74df6d-1a62-4219-87ff-959c52546c75","Type":"ContainerDied","Data":"1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40"} Nov 04 02:10:05.530548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.530456 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lnk5l" Nov 04 02:10:05.530548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.530461 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e467ebaf0cd65decacb464bda7d856af96063bba767ec62dc2519e0dc76bc40" Nov 04 02:10:05.652377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.652354 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:10:05.819174 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.819096 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\") pod \"7eb73e72-2fa2-4c98-bb5a-452923450590\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " Nov 04 02:10:05.819174 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.819144 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\" (UniqueName: \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\") pod \"7eb73e72-2fa2-4c98-bb5a-452923450590\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " Nov 04 02:10:05.819433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.819191 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\") pod \"7eb73e72-2fa2-4c98-bb5a-452923450590\" (UID: \"7eb73e72-2fa2-4c98-bb5a-452923450590\") " Nov 04 02:10:05.819824 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.819801 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle" pod "7eb73e72-2fa2-4c98-bb5a-452923450590" (UID: "7eb73e72-2fa2-4c98-bb5a-452923450590"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:05.821436 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.821412 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw" pod "7eb73e72-2fa2-4c98-bb5a-452923450590" (UID: "7eb73e72-2fa2-4c98-bb5a-452923450590"). InnerVolumeSpecName "kube-api-access-w7rzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:10:05.825221 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.825201 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util" pod "7eb73e72-2fa2-4c98-bb5a-452923450590" (UID: "7eb73e72-2fa2-4c98-bb5a-452923450590"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:10:05.919707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.919665 6073 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-util\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:05.919707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.919700 6073 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7eb73e72-2fa2-4c98-bb5a-452923450590-bundle\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:05.919707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:05.919714 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7rzw\" (UniqueName: \"kubernetes.io/projected/7eb73e72-2fa2-4c98-bb5a-452923450590-kube-api-access-w7rzw\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:10:06.281436 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281404 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-987dw"] Nov 04 02:10:06.281711 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281700 6073 cpu_manager.go:414] "containerMap: removing container" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="util" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281712 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="util" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281728 6073 cpu_manager.go:414] "containerMap: removing container" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="util" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281733 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="util" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281740 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="pull" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281745 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="pull" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281751 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="extract" Nov 04 02:10:06.281753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281755 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281766 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281771 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281776 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281781 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281790 6073 cpu_manager.go:414] "containerMap: removing container" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281795 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281802 6073 cpu_manager.go:414] "containerMap: removing container" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281806 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281812 6073 cpu_manager.go:414] "containerMap: removing container" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281816 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281822 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281826 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281831 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281835 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281840 6073 cpu_manager.go:414] "containerMap: removing container" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281845 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281852 6073 cpu_manager.go:414] "containerMap: removing container" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281856 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281861 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281866 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281872 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281876 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281887 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281891 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281896 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281901 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281909 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281913 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281920 6073 cpu_manager.go:414] "containerMap: removing container" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281923 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="util" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281928 6073 cpu_manager.go:414] "containerMap: removing container" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281932 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="extract" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281939 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281944 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281950 6073 cpu_manager.go:414] "containerMap: removing container" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281953 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281959 6073 cpu_manager.go:414] "containerMap: removing container" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281964 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281970 6073 cpu_manager.go:414] "containerMap: removing container" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="pull" Nov 04 02:10:06.281953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.281974 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="pull" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282017 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="a389607b-92e4-41d4-8145-3ca6d8b328a4" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282025 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="32e00307-c78b-4a5f-9852-089443996c1a" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282032 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="d01eb26c-f9c5-44b0-91a0-fe558a128257" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282037 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="7df1e2bb-ac92-4663-b21b-48b0d61bc17f" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282042 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f74df6d-1a62-4219-87ff-959c52546c75" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282048 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5415a7a-eefc-48ae-885d-eb323d150cd1" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282054 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="7eb73e72-2fa2-4c98-bb5a-452923450590" containerName="extract" Nov 04 02:10:06.283063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.282059 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cdde932-def9-4ff2-bf1a-ace0f4698008" containerName="extract" Nov 04 02:10:06.283860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.283844 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.336290 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.336259 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-wqvtc\"" Nov 04 02:10:06.336472 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.336368 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Nov 04 02:10:06.385548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.385512 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:10:06.386986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.386962 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-987dw"] Nov 04 02:10:06.425249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.425217 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/debf65d7-8889-4781-a7d5-6160cfaedfb3-tmp\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.425423 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.425264 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82w9f\" (UniqueName: \"kubernetes.io/projected/debf65d7-8889-4781-a7d5-6160cfaedfb3-kube-api-access-82w9f\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.525936 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.525910 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/debf65d7-8889-4781-a7d5-6160cfaedfb3-tmp\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.526106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.525946 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82w9f\" (UniqueName: \"kubernetes.io/projected/debf65d7-8889-4781-a7d5-6160cfaedfb3-kube-api-access-82w9f\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.526306 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.526288 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/debf65d7-8889-4781-a7d5-6160cfaedfb3-tmp\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.536182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.536122 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" Nov 04 02:10:06.536182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.536123 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cc572x" event={"ID":"7eb73e72-2fa2-4c98-bb5a-452923450590","Type":"ContainerDied","Data":"64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb"} Nov 04 02:10:06.536182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.536162 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64e68b824563adbdb2503bc3395467d02fa28cefa0198475443cc99db58e9cdb" Nov 04 02:10:06.586548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.586516 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82w9f\" (UniqueName: \"kubernetes.io/projected/debf65d7-8889-4781-a7d5-6160cfaedfb3-kube-api-access-82w9f\") pod \"jobset-operator-674cd59dc5-987dw\" (UID: \"debf65d7-8889-4781-a7d5-6160cfaedfb3\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.616540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.616513 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" Nov 04 02:10:06.983705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:06.983630 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-987dw"] Nov 04 02:10:06.986633 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:06.986609 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddebf65d7_8889_4781_a7d5_6160cfaedfb3.slice/crio-dcbb7a511665e0e19b4626eb0f27a7abc42aded8d3cf0a1dd5a6138e57277920 WatchSource:0}: Error finding container dcbb7a511665e0e19b4626eb0f27a7abc42aded8d3cf0a1dd5a6138e57277920: Status 404 returned error can't find the container with id dcbb7a511665e0e19b4626eb0f27a7abc42aded8d3cf0a1dd5a6138e57277920 Nov 04 02:10:07.542968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:07.542933 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" event={"ID":"debf65d7-8889-4781-a7d5-6160cfaedfb3","Type":"ContainerStarted","Data":"dcbb7a511665e0e19b4626eb0f27a7abc42aded8d3cf0a1dd5a6138e57277920"} Nov 04 02:10:10.558430 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:10.558391 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" event={"ID":"debf65d7-8889-4781-a7d5-6160cfaedfb3","Type":"ContainerStarted","Data":"8e17bb468810ae844413b4decfd04addaf6b5fd4c984d0be4df28c64b1001cd1"} Nov 04 02:10:10.674299 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:10.674247 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-987dw" podStartSLOduration=1.816767911 podStartE2EDuration="4.674232098s" totalImagesPullingTime="2.857464187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:10.672894422 +0000 UTC m=+188.368584621" watchObservedRunningTime="2026-11-04 02:10:10.674232098 +0000 UTC m=+188.369922299" Nov 04 02:10:17.844853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.844768 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr"] Nov 04 02:10:17.847186 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.847164 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.920973 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.920939 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.920973 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.920974 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.921172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.921011 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.921172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.921098 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.921172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.921117 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fznk\" (UniqueName: \"kubernetes.io/projected/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-kube-api-access-9fznk\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.921172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.921136 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:17.952524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952500 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 02:10:17.952524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952501 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 02:10:17.952524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952521 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 02:10:17.952733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952548 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-6rb58\"" Nov 04 02:10:17.952733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952567 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 02:10:17.952733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.952577 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 02:10:17.967152 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:17.967128 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr"] Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024718 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024766 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024822 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024868 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024892 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fznk\" (UniqueName: \"kubernetes.io/projected/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-kube-api-access-9fznk\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.025390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.024916 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.026508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.026482 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.028155 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.028133 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.028279 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.028193 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.028358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.028290 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.028419 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.028365 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.085925 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.085886 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fznk\" (UniqueName: \"kubernetes.io/projected/c4a7e85f-c1ca-4dde-b0ec-356142e9fb66-kube-api-access-9fznk\") pod \"opendatahub-operator-controller-manager-647c5cc759-4bzvr\" (UID: \"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.159456 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.159374 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:18.649161 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.649135 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr"] Nov 04 02:10:18.654646 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:18.654607 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a7e85f_c1ca_4dde_b0ec_356142e9fb66.slice/crio-5b1705c5b10f6a0cf4cd450c441ab0cb4f96a4e48967aca465efdfa19d52ba0b WatchSource:0}: Error finding container 5b1705c5b10f6a0cf4cd450c441ab0cb4f96a4e48967aca465efdfa19d52ba0b: Status 404 returned error can't find the container with id 5b1705c5b10f6a0cf4cd450c441ab0cb4f96a4e48967aca465efdfa19d52ba0b Nov 04 02:10:18.703550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.703521 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4"] Nov 04 02:10:18.706405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.706387 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:18.731139 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.731107 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7kfw\" (UniqueName: \"kubernetes.io/projected/8404641c-60cb-4359-b0a7-f61d203d16ea-kube-api-access-v7kfw\") pod \"authorino-operator-7b7bb4bc77-sp8n4\" (UID: \"8404641c-60cb-4359-b0a7-f61d203d16ea\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:18.761901 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.761870 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 02:10:18.762171 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.762155 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 02:10:18.762876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.762862 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mxp5x\"" Nov 04 02:10:18.767280 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.767260 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4"] Nov 04 02:10:18.831955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.831914 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7kfw\" (UniqueName: \"kubernetes.io/projected/8404641c-60cb-4359-b0a7-f61d203d16ea-kube-api-access-v7kfw\") pod \"authorino-operator-7b7bb4bc77-sp8n4\" (UID: \"8404641c-60cb-4359-b0a7-f61d203d16ea\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:18.896180 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:18.896149 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7kfw\" (UniqueName: \"kubernetes.io/projected/8404641c-60cb-4359-b0a7-f61d203d16ea-kube-api-access-v7kfw\") pod \"authorino-operator-7b7bb4bc77-sp8n4\" (UID: \"8404641c-60cb-4359-b0a7-f61d203d16ea\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:19.018377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:19.018323 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:19.395195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:19.395168 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4"] Nov 04 02:10:19.397314 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:19.397285 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8404641c_60cb_4359_b0a7_f61d203d16ea.slice/crio-24c777b399618f112a38df92de996c60b273df1704e7caaa3d830a23a0969eb0 WatchSource:0}: Error finding container 24c777b399618f112a38df92de996c60b273df1704e7caaa3d830a23a0969eb0: Status 404 returned error can't find the container with id 24c777b399618f112a38df92de996c60b273df1704e7caaa3d830a23a0969eb0 Nov 04 02:10:19.598740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:19.598704 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" event={"ID":"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66","Type":"ContainerStarted","Data":"5b1705c5b10f6a0cf4cd450c441ab0cb4f96a4e48967aca465efdfa19d52ba0b"} Nov 04 02:10:19.599962 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:19.599940 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" event={"ID":"8404641c-60cb-4359-b0a7-f61d203d16ea","Type":"ContainerStarted","Data":"24c777b399618f112a38df92de996c60b273df1704e7caaa3d830a23a0969eb0"} Nov 04 02:10:23.623543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:23.623446 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" event={"ID":"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66","Type":"ContainerStarted","Data":"8102b9617b5cc44b970cd293c91b3c2575840a967b7bece830235378df1abe4e"} Nov 04 02:10:23.625017 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:23.624990 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" event={"ID":"8404641c-60cb-4359-b0a7-f61d203d16ea","Type":"ContainerStarted","Data":"acbc7f1d4223f59fd1f411a9aea75baed733bac63bbfbbcd594f300c166d3df8"} Nov 04 02:10:23.625096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:23.625075 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:23.857274 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:10:23.857233 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:10:23.977905 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:23.977803 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" podStartSLOduration=2.369346986 podStartE2EDuration="5.977790087s" totalImagesPullingTime="3.608443101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:23.976786518 +0000 UTC m=+201.672476732" watchObservedRunningTime="2026-11-04 02:10:23.977790087 +0000 UTC m=+201.673480286" Nov 04 02:10:24.630956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:24.630925 6073 generic.go:356] "Generic (PLEG): container finished" podID="c4a7e85f-c1ca-4dde-b0ec-356142e9fb66" containerID="8102b9617b5cc44b970cd293c91b3c2575840a967b7bece830235378df1abe4e" exitCode=0 Nov 04 02:10:24.631371 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:24.631019 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" event={"ID":"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66","Type":"ContainerDied","Data":"8102b9617b5cc44b970cd293c91b3c2575840a967b7bece830235378df1abe4e"} Nov 04 02:10:25.636997 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:25.636964 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" event={"ID":"c4a7e85f-c1ca-4dde-b0ec-356142e9fb66","Type":"ContainerStarted","Data":"a89676bfc65a4d2f6847893a8e88d1db2e50fc09df634855f7e69a19821100d2"} Nov 04 02:10:25.637434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:25.637072 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:25.891796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:25.891742 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" podStartSLOduration=3.016327612 podStartE2EDuration="8.891713314s" totalImagesPullingTime="5.415418971s" totalInitContainerRuntime="459.966731ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:25.842271631 +0000 UTC m=+203.537961874" watchObservedRunningTime="2026-11-04 02:10:25.891713314 +0000 UTC m=+203.587403554" Nov 04 02:10:34.633552 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.633525 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sp8n4" Nov 04 02:10:34.807049 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.806947 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn"] Nov 04 02:10:34.811307 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.811284 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:34.865717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.865691 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8plk5\"" Nov 04 02:10:34.889352 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.889242 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t766j\" (UniqueName: \"kubernetes.io/projected/a304204a-1c02-4488-a994-c073e37fa54c-kube-api-access-t766j\") pod \"limitador-operator-controller-manager-5778bb4449-5x5qn\" (UID: \"a304204a-1c02-4488-a994-c073e37fa54c\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:34.929292 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.929259 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn"] Nov 04 02:10:34.990254 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:34.990223 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t766j\" (UniqueName: \"kubernetes.io/projected/a304204a-1c02-4488-a994-c073e37fa54c-kube-api-access-t766j\") pod \"limitador-operator-controller-manager-5778bb4449-5x5qn\" (UID: \"a304204a-1c02-4488-a994-c073e37fa54c\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:35.053813 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:35.053788 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t766j\" (UniqueName: \"kubernetes.io/projected/a304204a-1c02-4488-a994-c073e37fa54c-kube-api-access-t766j\") pod \"limitador-operator-controller-manager-5778bb4449-5x5qn\" (UID: \"a304204a-1c02-4488-a994-c073e37fa54c\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:35.124889 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:35.124849 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:35.498655 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:35.498636 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn"] Nov 04 02:10:35.501271 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:35.501245 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda304204a_1c02_4488_a994_c073e37fa54c.slice/crio-3a291dc53fabd38adf1cf872a801a936caf9bfe077da36cd8c18d465e0946fdb WatchSource:0}: Error finding container 3a291dc53fabd38adf1cf872a801a936caf9bfe077da36cd8c18d465e0946fdb: Status 404 returned error can't find the container with id 3a291dc53fabd38adf1cf872a801a936caf9bfe077da36cd8c18d465e0946fdb Nov 04 02:10:35.684580 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:35.684550 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" event={"ID":"a304204a-1c02-4488-a994-c073e37fa54c","Type":"ContainerStarted","Data":"3a291dc53fabd38adf1cf872a801a936caf9bfe077da36cd8c18d465e0946fdb"} Nov 04 02:10:36.644588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:36.644555 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4bzvr" Nov 04 02:10:38.700599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:38.700568 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" event={"ID":"a304204a-1c02-4488-a994-c073e37fa54c","Type":"ContainerStarted","Data":"ee13d64cab39f728b65d14ba2c62f52a4a2d54c934e432d94dc5d0fe4bdbb4b4"} Nov 04 02:10:38.701002 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:38.700701 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:38.820578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:38.820527 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" podStartSLOduration=2.174539677 podStartE2EDuration="4.820511079s" totalImagesPullingTime="2.645971402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:38.819363983 +0000 UTC m=+216.515054184" watchObservedRunningTime="2026-11-04 02:10:38.820511079 +0000 UTC m=+216.516201278" Nov 04 02:10:39.174776 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.174738 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw"] Nov 04 02:10:39.176909 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.176888 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.229164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.229128 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9ncl4\"" Nov 04 02:10:39.229164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.229150 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Nov 04 02:10:39.229364 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.229190 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Nov 04 02:10:39.235079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.235058 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw"] Nov 04 02:10:39.335090 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.335059 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wctj\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-kube-api-access-5wctj\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.335254 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.335098 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.435874 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.435802 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wctj\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-kube-api-access-5wctj\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.435874 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.435836 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.496383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.496355 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.496511 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.496395 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wctj\" (UniqueName: \"kubernetes.io/projected/820d2ee4-87a9-4746-870b-00add76a838b-kube-api-access-5wctj\") pod \"cert-manager-cainjector-5c57bbd4fc-6mvqw\" (UID: \"820d2ee4-87a9-4746-870b-00add76a838b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:39.787296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:39.787263 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" Nov 04 02:10:40.176078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:40.176054 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw"] Nov 04 02:10:40.710550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:40.710514 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" event={"ID":"820d2ee4-87a9-4746-870b-00add76a838b","Type":"ContainerStarted","Data":"c486e9a0887802a3c947a9b228009c97717f8cb3ea269c3c66372d897815dda0"} Nov 04 02:10:44.731901 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:44.731862 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" event={"ID":"820d2ee4-87a9-4746-870b-00add76a838b","Type":"ContainerStarted","Data":"1c69f609fb5f4418ad579639896cc17396d15f64d5c477888664336837a99bdd"} Nov 04 02:10:44.847259 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:44.847200 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-6mvqw" podStartSLOduration=2.229468737 podStartE2EDuration="5.847184738s" totalImagesPullingTime="3.617716001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:44.845822323 +0000 UTC m=+222.541512524" watchObservedRunningTime="2026-11-04 02:10:44.847184738 +0000 UTC m=+222.542874937" Nov 04 02:10:49.707264 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:49.707235 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-5x5qn" Nov 04 02:10:50.099982 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.099938 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr"] Nov 04 02:10:50.103668 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.103641 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.156866 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.156837 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 02:10:50.156866 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.156842 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pc4n4\"" Nov 04 02:10:50.157115 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.156973 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 02:10:50.170097 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.170067 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr"] Nov 04 02:10:50.246401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.246358 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f22c973b-668b-4707-a5eb-5673763e5e6e-tmp\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.246587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.246439 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7rs\" (UniqueName: \"kubernetes.io/projected/f22c973b-668b-4707-a5eb-5673763e5e6e-kube-api-access-xw7rs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.246587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.246498 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f22c973b-668b-4707-a5eb-5673763e5e6e-tls-certs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.347313 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.347274 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7rs\" (UniqueName: \"kubernetes.io/projected/f22c973b-668b-4707-a5eb-5673763e5e6e-kube-api-access-xw7rs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.347512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.347345 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f22c973b-668b-4707-a5eb-5673763e5e6e-tls-certs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.347512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.347397 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f22c973b-668b-4707-a5eb-5673763e5e6e-tmp\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.349842 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.349814 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f22c973b-668b-4707-a5eb-5673763e5e6e-tmp\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.350049 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.350008 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f22c973b-668b-4707-a5eb-5673763e5e6e-tls-certs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.411154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.411123 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7rs\" (UniqueName: \"kubernetes.io/projected/f22c973b-668b-4707-a5eb-5673763e5e6e-kube-api-access-xw7rs\") pod \"kube-auth-proxy-56b54d794b-t6xrr\" (UID: \"f22c973b-668b-4707-a5eb-5673763e5e6e\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.415964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.415944 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" Nov 04 02:10:50.732254 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.732154 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d"] Nov 04 02:10:50.737848 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.737820 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.800409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.800378 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d"] Nov 04 02:10:50.846388 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.846357 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cv55m\"" Nov 04 02:10:50.849366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.849345 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr"] Nov 04 02:10:50.850905 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:50.850876 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22c973b_668b_4707_a5eb_5673763e5e6e.slice/crio-ba41ca10a6d6e8850104d28f00b2fde0d3315e309e26aae2267ed6961fd4055a WatchSource:0}: Error finding container ba41ca10a6d6e8850104d28f00b2fde0d3315e309e26aae2267ed6961fd4055a: Status 404 returned error can't find the container with id ba41ca10a6d6e8850104d28f00b2fde0d3315e309e26aae2267ed6961fd4055a Nov 04 02:10:50.851273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851250 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851386 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851297 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851476 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851445 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851525 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851555 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851572 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079bd183-155c-40e7-ab79-41757399ccc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851656 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851589 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079bd183-155c-40e7-ab79-41757399ccc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851656 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851605 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2nd4\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-kube-api-access-m2nd4\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.851797 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.851777 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952625 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952592 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952640 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952690 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952709 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952732 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079bd183-155c-40e7-ab79-41757399ccc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.952812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.952757 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079bd183-155c-40e7-ab79-41757399ccc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953044 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2nd4\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-kube-api-access-m2nd4\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953087 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953109 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953228 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953153 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953635 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953548 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953635 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953565 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.953910 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.953886 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.954034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.954010 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079bd183-155c-40e7-ab79-41757399ccc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.955377 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.955326 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079bd183-155c-40e7-ab79-41757399ccc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:50.955578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:50.955556 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079bd183-155c-40e7-ab79-41757399ccc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:51.019070 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.019043 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:51.034976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.019201 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2nd4\" (UniqueName: \"kubernetes.io/projected/079bd183-155c-40e7-ab79-41757399ccc5-kube-api-access-m2nd4\") pod \"data-science-gateway-data-science-gateway-class-549f9d654fjff5d\" (UID: \"079bd183-155c-40e7-ab79-41757399ccc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:51.052001 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.051974 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:51.477509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.477482 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d"] Nov 04 02:10:51.478937 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:10:51.478904 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079bd183_155c_40e7_ab79_41757399ccc5.slice/crio-84afe397e72c538e1a2bd8d2cf1ae70f07a7277f410332e1311086c491b23a1f WatchSource:0}: Error finding container 84afe397e72c538e1a2bd8d2cf1ae70f07a7277f410332e1311086c491b23a1f: Status 404 returned error can't find the container with id 84afe397e72c538e1a2bd8d2cf1ae70f07a7277f410332e1311086c491b23a1f Nov 04 02:10:51.481245 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.481214 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:10:51.481322 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.481277 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:10:51.481322 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.481305 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:10:51.766792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.766747 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" event={"ID":"f22c973b-668b-4707-a5eb-5673763e5e6e","Type":"ContainerStarted","Data":"ba41ca10a6d6e8850104d28f00b2fde0d3315e309e26aae2267ed6961fd4055a"} Nov 04 02:10:51.768466 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.768439 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" event={"ID":"079bd183-155c-40e7-ab79-41757399ccc5","Type":"ContainerStarted","Data":"1a5f2fcf1df71256840fa7ea2cae40924dc4e63392cac091361d4f1d96fd4d73"} Nov 04 02:10:51.768598 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:51.768484 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" event={"ID":"079bd183-155c-40e7-ab79-41757399ccc5","Type":"ContainerStarted","Data":"84afe397e72c538e1a2bd8d2cf1ae70f07a7277f410332e1311086c491b23a1f"} Nov 04 02:10:52.053091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:52.053002 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:52.059465 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:52.059437 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:52.185446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:52.185390 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" podStartSLOduration=2.185376313 podStartE2EDuration="2.185376313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:51.89102605 +0000 UTC m=+229.586716249" watchObservedRunningTime="2026-11-04 02:10:52.185376313 +0000 UTC m=+229.881066513" Nov 04 02:10:52.774407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:52.774369 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:52.775532 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:52.775513 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-549f9d654fjff5d" Nov 04 02:10:53.863455 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:10:53.863417 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:10:54.786937 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:54.786895 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" event={"ID":"f22c973b-668b-4707-a5eb-5673763e5e6e","Type":"ContainerStarted","Data":"a399cb15a4d9d9dc2bc6c58b9fc2a16cc1ecc63322b4112a5a3963dfc0166d80"} Nov 04 02:10:54.906625 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:10:54.906566 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56b54d794b-t6xrr" podStartSLOduration=1.424228757 podStartE2EDuration="4.906551868s" totalImagesPullingTime="3.482323111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:54.905142866 +0000 UTC m=+232.600833087" watchObservedRunningTime="2026-11-04 02:10:54.906551868 +0000 UTC m=+232.602242069" Nov 04 02:11:23.857728 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:23.857691 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:11:44.918076 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.917993 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4"] Nov 04 02:11:44.923495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.923461 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:44.968018 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.967987 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:44.968018 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.968024 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:44.968219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.968052 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:44.968219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.968191 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmgz7\" (UniqueName: \"kubernetes.io/projected/7fc15fae-0042-4cb1-b873-1f4a676b976f-kube-api-access-kmgz7\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:44.977780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:44.977734 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Nov 04 02:11:45.027213 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.027184 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-q5sf9\"" Nov 04 02:11:45.027511 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.027493 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Nov 04 02:11:45.027511 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.027492 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Nov 04 02:11:45.027677 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.027664 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 02:11:45.027725 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.027667 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 02:11:45.029110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.029092 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4"] Nov 04 02:11:45.069370 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.069318 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.069512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.069384 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.069512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.069463 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmgz7\" (UniqueName: \"kubernetes.io/projected/7fc15fae-0042-4cb1-b873-1f4a676b976f-kube-api-access-kmgz7\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.069512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.069498 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.069616 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:45.069544 6073 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found Nov 04 02:11:45.069616 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:45.069560 6073 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found Nov 04 02:11:45.069676 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:45.069616 6073 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert podName:7fc15fae-0042-4cb1-b873-1f4a676b976f nodeName:}" failed. No retries permitted until 2026-11-04 02:11:45.569598822 +0000 UTC m=+283.265289016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" (UID: "7fc15fae-0042-4cb1-b873-1f4a676b976f") : secret "usage-logs-tenancy-proxy-tls" not found Nov 04 02:11:45.069676 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:45.069634 6073 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source podName:7fc15fae-0042-4cb1-b873-1f4a676b976f nodeName:}" failed. No retries permitted until 2026-11-04 02:11:45.569627789 +0000 UTC m=+283.265318009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" (UID: "7fc15fae-0042-4cb1-b873-1f4a676b976f") : configmap "usage-tenancy-proxy-source" not found Nov 04 02:11:45.070031 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.070014 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.130669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.130640 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmgz7\" (UniqueName: \"kubernetes.io/projected/7fc15fae-0042-4cb1-b873-1f4a676b976f-kube-api-access-kmgz7\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.574418 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.574380 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.574600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.574429 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.575229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.575201 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7fc15fae-0042-4cb1-b873-1f4a676b976f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.576991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.576969 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7fc15fae-0042-4cb1-b873-1f4a676b976f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-nwmb4\" (UID: \"7fc15fae-0042-4cb1-b873-1f4a676b976f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:45.836958 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:45.836873 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:11:46.209439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:46.209412 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4"] Nov 04 02:11:46.211812 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:11:46.211779 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc15fae_0042_4cb1_b873_1f4a676b976f.slice/crio-07b3994e08af7619ae33f607676dbf355f0f5af656046c3d74d167177d2550e6 WatchSource:0}: Error finding container 07b3994e08af7619ae33f607676dbf355f0f5af656046c3d74d167177d2550e6: Status 404 returned error can't find the container with id 07b3994e08af7619ae33f607676dbf355f0f5af656046c3d74d167177d2550e6 Nov 04 02:11:47.040003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:11:47.039966 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" event={"ID":"7fc15fae-0042-4cb1-b873-1f4a676b976f","Type":"ContainerStarted","Data":"07b3994e08af7619ae33f607676dbf355f0f5af656046c3d74d167177d2550e6"} Nov 04 02:11:53.865324 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:11:53.865285 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:12:00.118426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:00.118385 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" event={"ID":"7fc15fae-0042-4cb1-b873-1f4a676b976f","Type":"ContainerStarted","Data":"62f545096a1f8b633d812cde643ba90ebe73a459f2e81d61ce3289b92abbbd1f"} Nov 04 02:12:00.118839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:00.118488 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:12:00.236106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:00.235985 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" podStartSLOduration=2.499513256 podStartE2EDuration="16.235967334s" totalImagesPullingTime="13.736454078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:00.234375277 +0000 UTC m=+297.930065477" watchObservedRunningTime="2026-11-04 02:12:00.235967334 +0000 UTC m=+297.931657535" Nov 04 02:12:09.372458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.372425 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-24ksh"] Nov 04 02:12:09.378690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.378661 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.433431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.433401 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-24ksh"] Nov 04 02:12:09.481431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.481399 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Nov 04 02:12:09.481431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.481397 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5wzqb\"" Nov 04 02:12:09.481632 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.481452 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Nov 04 02:12:09.517564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.517534 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf9m7\" (UniqueName: \"kubernetes.io/projected/dadde5dc-97f2-479d-baa2-5f8b227aac36-kube-api-access-xf9m7\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.517743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.517574 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-metrics-certs\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.517811 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.517765 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-cert\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.618543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.618513 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-cert\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.618735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.618590 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf9m7\" (UniqueName: \"kubernetes.io/projected/dadde5dc-97f2-479d-baa2-5f8b227aac36-kube-api-access-xf9m7\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.618735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.618620 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-metrics-certs\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.621139 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.621111 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-metrics-certs\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.621251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.621222 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dadde5dc-97f2-479d-baa2-5f8b227aac36-cert\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.679851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.679773 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf9m7\" (UniqueName: \"kubernetes.io/projected/dadde5dc-97f2-479d-baa2-5f8b227aac36-kube-api-access-xf9m7\") pod \"odh-model-controller-769875db56-24ksh\" (UID: \"dadde5dc-97f2-479d-baa2-5f8b227aac36\") " pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:09.691054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:09.691023 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:10.062396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:10.062371 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-24ksh"] Nov 04 02:12:10.064833 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:12:10.064807 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddadde5dc_97f2_479d_baa2_5f8b227aac36.slice/crio-09ccb62a3d018ef1ce8172520aca06e44c941381f8e0c228a6b8a83bb16c0a27 WatchSource:0}: Error finding container 09ccb62a3d018ef1ce8172520aca06e44c941381f8e0c228a6b8a83bb16c0a27: Status 404 returned error can't find the container with id 09ccb62a3d018ef1ce8172520aca06e44c941381f8e0c228a6b8a83bb16c0a27 Nov 04 02:12:10.172419 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:10.172389 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-24ksh" event={"ID":"dadde5dc-97f2-479d-baa2-5f8b227aac36","Type":"ContainerStarted","Data":"09ccb62a3d018ef1ce8172520aca06e44c941381f8e0c228a6b8a83bb16c0a27"} Nov 04 02:12:11.129291 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:11.129259 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4" Nov 04 02:12:14.193839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:14.193805 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-24ksh" event={"ID":"dadde5dc-97f2-479d-baa2-5f8b227aac36","Type":"ContainerStarted","Data":"3e6e08d1ec8893adaedddd65a5566f4a657a0ecc3aafbe33c364e25170f9da2a"} Nov 04 02:12:14.194251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:14.193932 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:14.316348 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:14.316275 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-24ksh" podStartSLOduration=2.138721089 podStartE2EDuration="5.316261388s" totalImagesPullingTime="3.177540299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:14.314985688 +0000 UTC m=+312.010675889" watchObservedRunningTime="2026-11-04 02:12:14.316261388 +0000 UTC m=+312.011951604" Nov 04 02:12:17.168409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.168377 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:17.171941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.171924 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.189841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.189816 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.189959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.189858 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.189959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.189897 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.190035 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.190016 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shp4b\" (UniqueName: \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.273847 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.273815 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 02:12:17.273847 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.273815 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 02:12:17.273847 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.273823 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p956t\"" Nov 04 02:12:17.274089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.273866 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:12:17.274089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.273897 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:12:17.275916 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.275900 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:17.290852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.290826 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.290964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.290876 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.290964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.290931 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shp4b\" (UniqueName: \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.290964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.290949 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.291661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.291637 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.293469 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.293446 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.293927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.293901 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.352000 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.351963 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shp4b\" (UniqueName: \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\") pod \"maas-api-674d648f7b-488f4\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.485022 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.484933 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:17.891162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:17.891141 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:17.893234 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:12:17.893207 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75782b9_c7a7_4d16_9fa7_489a60ea9bd5.slice/crio-5b4157b9b609de01e9d41ccbde0833c390e8024112c107c1621dbbf734a3946d WatchSource:0}: Error finding container 5b4157b9b609de01e9d41ccbde0833c390e8024112c107c1621dbbf734a3946d: Status 404 returned error can't find the container with id 5b4157b9b609de01e9d41ccbde0833c390e8024112c107c1621dbbf734a3946d Nov 04 02:12:18.217553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:18.217475 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" event={"ID":"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5","Type":"ContainerStarted","Data":"5b4157b9b609de01e9d41ccbde0833c390e8024112c107c1621dbbf734a3946d"} Nov 04 02:12:23.856829 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:12:23.856787 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:12:24.249774 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:24.249693 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" event={"ID":"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5","Type":"ContainerStarted","Data":"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d"} Nov 04 02:12:24.249927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:24.249816 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:24.367914 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:24.367868 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" podStartSLOduration=1.859745399 podStartE2EDuration="7.367854215s" totalImagesPullingTime="5.508108816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:24.366666263 +0000 UTC m=+322.062356465" watchObservedRunningTime="2026-11-04 02:12:24.367854215 +0000 UTC m=+322.063544415" Nov 04 02:12:25.202485 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:25.202457 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-24ksh" Nov 04 02:12:30.260231 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:30.260202 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:41.365686 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.365652 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:41.366201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.365857 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" podUID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" containerName="maas-api" containerID="cri-o://963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d" gracePeriod=30 Nov 04 02:12:41.614867 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.614844 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:41.723569 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.723489 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\") pod \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " Nov 04 02:12:41.723722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.723582 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\" (UniqueName: \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\") pod \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " Nov 04 02:12:41.723722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.723618 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\") pod \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " Nov 04 02:12:41.723722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.723655 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\") pod \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\" (UID: \"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5\") " Nov 04 02:12:41.724053 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.724020 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca" pod "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" (UID: "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:12:41.725868 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.725841 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b" pod "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" (UID: "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5"). InnerVolumeSpecName "kube-api-access-shp4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:12:41.725959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.725873 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls" pod "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" (UID: "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:12:41.725959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.725923 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs" pod "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" (UID: "c75782b9-c7a7-4d16-9fa7-489a60ea9bd5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:12:41.825116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.825084 6073 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-openshift-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:12:41.825116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.825111 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-maas-api-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:12:41.825116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.825121 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shp4b\" (UniqueName: \"kubernetes.io/projected/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-kube-api-access-shp4b\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:12:41.825364 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:41.825132 6073 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5-metrics-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:12:42.338128 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.338095 6073 generic.go:356] "Generic (PLEG): container finished" podID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" containerID="963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d" exitCode=0 Nov 04 02:12:42.338304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.338146 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" event={"ID":"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5","Type":"ContainerDied","Data":"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d"} Nov 04 02:12:42.338304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.338182 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" event={"ID":"c75782b9-c7a7-4d16-9fa7-489a60ea9bd5","Type":"ContainerDied","Data":"5b4157b9b609de01e9d41ccbde0833c390e8024112c107c1621dbbf734a3946d"} Nov 04 02:12:42.338304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.338204 6073 scope.go:122] "RemoveContainer" containerID="963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d" Nov 04 02:12:42.338304 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.338146 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-488f4" Nov 04 02:12:42.347636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.347618 6073 scope.go:122] "RemoveContainer" containerID="963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d" Nov 04 02:12:42.347906 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:12:42.347878 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d\": container with ID starting with 963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d not found: ID does not exist" containerID="963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d" Nov 04 02:12:42.347995 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.347914 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d"} err="failed to get container status \"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d\": rpc error: code = NotFound desc = could not find container \"963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d\": container with ID starting with 963b8d231c8830eefc6faad7c5e3b8e0ed29fbc9e99f7eec875e92fa76f7553d not found: ID does not exist" Nov 04 02:12:42.500920 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.500887 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:42.550587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:42.550551 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-488f4"] Nov 04 02:12:44.424835 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:12:44.424798 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" path="/var/lib/kubelet/pods/c75782b9-c7a7-4d16-9fa7-489a60ea9bd5/volumes" Nov 04 02:12:53.857627 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:12:53.857592 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:13:23.858051 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:13:23.857972 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:13:53.856610 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:13:53.856567 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:14:09.131178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.131136 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc"] Nov 04 02:14:09.131827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.131718 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" containerName="maas-api" Nov 04 02:14:09.131827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.131739 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" containerName="maas-api" Nov 04 02:14:09.131952 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.131859 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c75782b9-c7a7-4d16-9fa7-489a60ea9bd5" containerName="maas-api" Nov 04 02:14:09.136641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.136619 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.194130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.194095 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc"] Nov 04 02:14:09.212879 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.212847 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.213042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.212890 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk654\" (UniqueName: \"kubernetes.io/projected/c2b6848b-96c0-404c-b80e-9b3dd515ac64-kube-api-access-vk654\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.213042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.212920 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.213135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.213047 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.213135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.213083 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.213135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.213105 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.241115 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.241089 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Nov 04 02:14:09.241115 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.241108 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Nov 04 02:14:09.241281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.241095 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6f7xg\"" Nov 04 02:14:09.241281 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.241198 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Nov 04 02:14:09.314267 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314234 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314459 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314278 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk654\" (UniqueName: \"kubernetes.io/projected/c2b6848b-96c0-404c-b80e-9b3dd515ac64-kube-api-access-vk654\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314459 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314310 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314459 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314420 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314458 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314488 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314789 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314796 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.314900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.314838 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.316547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.316527 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b6848b-96c0-404c-b80e-9b3dd515ac64-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.316858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.316841 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b6848b-96c0-404c-b80e-9b3dd515ac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.374816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.374784 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk654\" (UniqueName: \"kubernetes.io/projected/c2b6848b-96c0-404c-b80e-9b3dd515ac64-kube-api-access-vk654\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-r2rjc\" (UID: \"c2b6848b-96c0-404c-b80e-9b3dd515ac64\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.449767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.449686 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:09.822159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:09.822118 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc"] Nov 04 02:14:09.825411 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:09.825381 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b6848b_96c0_404c_b80e_9b3dd515ac64.slice/crio-2866156a46c0f855a26d53c0481acfc6509929a11d866e7a7699a96400456a85 WatchSource:0}: Error finding container 2866156a46c0f855a26d53c0481acfc6509929a11d866e7a7699a96400456a85: Status 404 returned error can't find the container with id 2866156a46c0f855a26d53c0481acfc6509929a11d866e7a7699a96400456a85 Nov 04 02:14:10.765004 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:10.764970 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" event={"ID":"c2b6848b-96c0-404c-b80e-9b3dd515ac64","Type":"ContainerStarted","Data":"2866156a46c0f855a26d53c0481acfc6509929a11d866e7a7699a96400456a85"} Nov 04 02:14:12.778972 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:12.778933 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" event={"ID":"c2b6848b-96c0-404c-b80e-9b3dd515ac64","Type":"ContainerStarted","Data":"d9dbf375c21bfefa03387c5b4f28045a1f339ffcb59b948c06aef2875e19238f"} Nov 04 02:14:12.897063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:12.897006 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" podStartSLOduration=1.755788143 podStartE2EDuration="3.896991192s" totalImagesPullingTime="2.141203049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:12.895797009 +0000 UTC m=+430.591487210" watchObservedRunningTime="2026-11-04 02:14:12.896991192 +0000 UTC m=+430.592681391" Nov 04 02:14:13.450513 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:13.450477 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:17.184110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.184077 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-8m2jn"] Nov 04 02:14:17.187729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.187710 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289062 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289021 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" Nov 04 02:14:17.289630 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289603 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289698 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289645 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmsx\" (UniqueName: \"kubernetes.io/projected/e49e3b05-2a7b-409a-b45f-e0060cf492ea-kube-api-access-vcmsx\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289698 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289678 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289810 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289780 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289847 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289841 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.289958 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.289942 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.291130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.291115 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-8m2jn"] Nov 04 02:14:17.390768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.390731 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.390768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.390772 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmsx\" (UniqueName: \"kubernetes.io/projected/e49e3b05-2a7b-409a-b45f-e0060cf492ea-kube-api-access-vcmsx\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.390991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.390880 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.390991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.390926 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.391071 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.390996 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.391119 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.391066 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.391240 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.391215 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.391318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.391248 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.391465 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.391442 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.393361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.393315 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e49e3b05-2a7b-409a-b45f-e0060cf492ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.393530 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.393513 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e49e3b05-2a7b-409a-b45f-e0060cf492ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.454670 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.454599 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmsx\" (UniqueName: \"kubernetes.io/projected/e49e3b05-2a7b-409a-b45f-e0060cf492ea-kube-api-access-vcmsx\") pod \"echo-simulator-kserve-64b6f8f96c-8m2jn\" (UID: \"e49e3b05-2a7b-409a-b45f-e0060cf492ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.500376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.500347 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:17.879509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:17.879485 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-8m2jn"] Nov 04 02:14:17.881970 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:17.881936 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49e3b05_2a7b_409a_b45f_e0060cf492ea.slice/crio-ba0aed55d4253e2b367843ba51ddf83c5b64bb5ac659b41eeb47757bd4714093 WatchSource:0}: Error finding container ba0aed55d4253e2b367843ba51ddf83c5b64bb5ac659b41eeb47757bd4714093: Status 404 returned error can't find the container with id ba0aed55d4253e2b367843ba51ddf83c5b64bb5ac659b41eeb47757bd4714093 Nov 04 02:14:18.808738 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:18.808706 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" event={"ID":"e49e3b05-2a7b-409a-b45f-e0060cf492ea","Type":"ContainerStarted","Data":"ba0aed55d4253e2b367843ba51ddf83c5b64bb5ac659b41eeb47757bd4714093"} Nov 04 02:14:19.450595 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:19.450550 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:19.463664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:19.463636 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:19.825732 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:19.825704 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-r2rjc" Nov 04 02:14:21.826904 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:21.826869 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" event={"ID":"e49e3b05-2a7b-409a-b45f-e0060cf492ea","Type":"ContainerStarted","Data":"9598822336db5b13a7f34e0eb5cf183836c833fec318533651bb3b8d226882c8"} Nov 04 02:14:21.945403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:21.945307 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" podStartSLOduration=1.839583661 podStartE2EDuration="4.94529242s" totalImagesPullingTime="3.105708759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:21.94380861 +0000 UTC m=+439.639498809" watchObservedRunningTime="2026-11-04 02:14:21.94529242 +0000 UTC m=+439.640982661" Nov 04 02:14:22.500659 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:22.500624 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:23.857420 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:14:23.857321 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:14:24.265736 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.265710 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72"] Nov 04 02:14:24.269468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.269449 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.358855 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.358822 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aecb1401-03fc-432e-b009-c1e05ce65482-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.359015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.358880 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.359015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.358961 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fthm5\" (UniqueName: \"kubernetes.io/projected/aecb1401-03fc-432e-b009-c1e05ce65482-kube-api-access-fthm5\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.359091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.359023 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.359125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.359088 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.359156 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.359133 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.370786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.370765 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Nov 04 02:14:24.372543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.372523 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72"] Nov 04 02:14:24.460203 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460169 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460203 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460206 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460228 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460263 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aecb1401-03fc-432e-b009-c1e05ce65482-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460303 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460324 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fthm5\" (UniqueName: \"kubernetes.io/projected/aecb1401-03fc-432e-b009-c1e05ce65482-kube-api-access-fthm5\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460718 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460653 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460728 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.460780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.460760 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.462668 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.462645 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aecb1401-03fc-432e-b009-c1e05ce65482-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.462931 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.462911 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aecb1401-03fc-432e-b009-c1e05ce65482-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.521133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.521061 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fthm5\" (UniqueName: \"kubernetes.io/projected/aecb1401-03fc-432e-b009-c1e05ce65482-kube-api-access-fthm5\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72\" (UID: \"aecb1401-03fc-432e-b009-c1e05ce65482\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.580459 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.580422 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:24.954903 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:24.954877 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72"] Nov 04 02:14:24.957415 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:24.957382 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaecb1401_03fc_432e_b009_c1e05ce65482.slice/crio-02e79944bd8e85da901c421f8637d9dc096322abb97407aa89c9fbd560ec21e9 WatchSource:0}: Error finding container 02e79944bd8e85da901c421f8637d9dc096322abb97407aa89c9fbd560ec21e9: Status 404 returned error can't find the container with id 02e79944bd8e85da901c421f8637d9dc096322abb97407aa89c9fbd560ec21e9 Nov 04 02:14:25.852790 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:25.852699 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" event={"ID":"aecb1401-03fc-432e-b009-c1e05ce65482","Type":"ContainerStarted","Data":"a570c8ac0dbe27aa6132617abd36c7e7be2b54344c56d3d3b056af9c958867fd"} Nov 04 02:14:25.852790 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:25.852747 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" event={"ID":"aecb1401-03fc-432e-b009-c1e05ce65482","Type":"ContainerStarted","Data":"02e79944bd8e85da901c421f8637d9dc096322abb97407aa89c9fbd560ec21e9"} Nov 04 02:14:25.970787 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:25.970736 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" podStartSLOduration=1.375287375 podStartE2EDuration="1.970721017s" totalImagesPullingTime="595.433642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:25.969376346 +0000 UTC m=+443.665066546" watchObservedRunningTime="2026-11-04 02:14:25.970721017 +0000 UTC m=+443.666411216" Nov 04 02:14:26.581005 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:26.580972 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:27.501470 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.501436 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:27.514517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.514490 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:27.874175 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.874149 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf"] Nov 04 02:14:27.877751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.877734 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-8m2jn" Nov 04 02:14:27.877941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.877930 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.930217 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.930193 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Nov 04 02:14:27.936577 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.936553 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf"] Nov 04 02:14:27.994968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.994937 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.994968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.994967 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.995170 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.995061 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.995236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.995166 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj8ct\" (UniqueName: \"kubernetes.io/projected/41757265-bd34-464f-8420-61e0771a36c5-kube-api-access-zj8ct\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.995236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.995198 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:27.995366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:27.995280 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41757265-bd34-464f-8420-61e0771a36c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096221 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096187 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096221 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096218 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096257 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096304 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj8ct\" (UniqueName: \"kubernetes.io/projected/41757265-bd34-464f-8420-61e0771a36c5-kube-api-access-zj8ct\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096358 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096489 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096382 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41757265-bd34-464f-8420-61e0771a36c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096737 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096701 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096758 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.096796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.096780 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.098788 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.098759 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41757265-bd34-464f-8420-61e0771a36c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.099025 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.099010 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41757265-bd34-464f-8420-61e0771a36c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.157157 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.157091 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj8ct\" (UniqueName: \"kubernetes.io/projected/41757265-bd34-464f-8420-61e0771a36c5-kube-api-access-zj8ct\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf\" (UID: \"41757265-bd34-464f-8420-61e0771a36c5\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.190033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.190006 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:28.565415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.565390 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf"] Nov 04 02:14:28.567255 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:28.567232 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41757265_bd34_464f_8420_61e0771a36c5.slice/crio-7475f5d9e414072cf1114fd8fa3eded82e0b9ec9603cf2d526789d3aef0dd84b WatchSource:0}: Error finding container 7475f5d9e414072cf1114fd8fa3eded82e0b9ec9603cf2d526789d3aef0dd84b: Status 404 returned error can't find the container with id 7475f5d9e414072cf1114fd8fa3eded82e0b9ec9603cf2d526789d3aef0dd84b Nov 04 02:14:28.568751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.568734 6073 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:14:28.869826 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:28.869748 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" event={"ID":"41757265-bd34-464f-8420-61e0771a36c5","Type":"ContainerStarted","Data":"7475f5d9e414072cf1114fd8fa3eded82e0b9ec9603cf2d526789d3aef0dd84b"} Nov 04 02:14:29.876296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:29.876258 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" event={"ID":"41757265-bd34-464f-8420-61e0771a36c5","Type":"ContainerStarted","Data":"c3ff049ccf7868f45115fb676bcb53aa00691dbe5688177ab4afcf4ecbd782ab"} Nov 04 02:14:29.993892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:29.993840 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" podStartSLOduration=2.4032709150000002 podStartE2EDuration="2.993826141s" totalImagesPullingTime="590.555226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:29.993503308 +0000 UTC m=+447.689193509" watchObservedRunningTime="2026-11-04 02:14:29.993826141 +0000 UTC m=+447.689516340" Nov 04 02:14:30.191002 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:30.190928 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:34.580608 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:34.580564 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:34.592872 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:34.592850 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:34.916151 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:34.916081 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72" Nov 04 02:14:38.190751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:38.190721 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:38.203102 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:38.203071 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:38.937269 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:38.937245 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf" Nov 04 02:14:39.245695 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.245622 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb"] Nov 04 02:14:39.249276 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.249259 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.296735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.296710 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.296856 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.296754 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42d3046d-924e-4072-9f00-135939bbea7c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.296856 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.296831 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.296856 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.296853 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwxns\" (UniqueName: \"kubernetes.io/projected/42d3046d-924e-4072-9f00-135939bbea7c-kube-api-access-fwxns\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.296977 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.296887 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.297015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.297008 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.301429 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.301405 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Nov 04 02:14:39.307470 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.307448 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb"] Nov 04 02:14:39.398175 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398073 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398365 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398227 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42d3046d-924e-4072-9f00-135939bbea7c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398365 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398276 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398365 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398303 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwxns\" (UniqueName: \"kubernetes.io/projected/42d3046d-924e-4072-9f00-135939bbea7c-kube-api-access-fwxns\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398367 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398400 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398442 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398703 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398669 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.398703 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.398697 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.400660 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.400637 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42d3046d-924e-4072-9f00-135939bbea7c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.400881 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.400867 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42d3046d-924e-4072-9f00-135939bbea7c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.461318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.461285 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwxns\" (UniqueName: \"kubernetes.io/projected/42d3046d-924e-4072-9f00-135939bbea7c-kube-api-access-fwxns\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-wlwzb\" (UID: \"42d3046d-924e-4072-9f00-135939bbea7c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.561506 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.561475 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:39.938482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:39.938455 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb"] Nov 04 02:14:39.940308 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:39.940280 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d3046d_924e_4072_9f00_135939bbea7c.slice/crio-89824ebfa7c45a174455ef84f8717b51f070a3af79a9bc011b7adabeff5b6387 WatchSource:0}: Error finding container 89824ebfa7c45a174455ef84f8717b51f070a3af79a9bc011b7adabeff5b6387: Status 404 returned error can't find the container with id 89824ebfa7c45a174455ef84f8717b51f070a3af79a9bc011b7adabeff5b6387 Nov 04 02:14:40.944828 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:40.944748 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" event={"ID":"42d3046d-924e-4072-9f00-135939bbea7c","Type":"ContainerStarted","Data":"989a2becdecc870af56e6afd24ca2c76f06dbbe329237fcf9d3654baad82724b"} Nov 04 02:14:40.944828 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:40.944800 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" event={"ID":"42d3046d-924e-4072-9f00-135939bbea7c","Type":"ContainerStarted","Data":"89824ebfa7c45a174455ef84f8717b51f070a3af79a9bc011b7adabeff5b6387"} Nov 04 02:14:41.062636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:41.062585 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" podStartSLOduration=1.478780199 podStartE2EDuration="2.062570438s" totalImagesPullingTime="583.790239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:41.061804648 +0000 UTC m=+458.757494850" watchObservedRunningTime="2026-11-04 02:14:41.062570438 +0000 UTC m=+458.758260637" Nov 04 02:14:41.561839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:41.561803 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:49.562522 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:49.562488 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:49.574756 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:49.574729 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:49.999199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:49.999121 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-wlwzb" Nov 04 02:14:51.084379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.084346 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:14:51.089480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.089463 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.134172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.134146 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:14:51.137680 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.137666 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.149882 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.149863 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vnp8f\"" Nov 04 02:14:51.191990 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.191963 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:14:51.210432 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210407 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210439 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210474 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.210628 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210585 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.210628 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210619 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210701 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.210764 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210745 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210822 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210809 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.210855 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210836 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210940 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210922 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.210973 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210964 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.211008 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.210993 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.211039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211022 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tjvz\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.211093 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211080 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfd6j\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.211139 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211125 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.211174 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211145 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.211174 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211162 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.211289 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.211187 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.241562 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.241530 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:14:51.312172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312147 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tjvz\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312184 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfd6j\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312212 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312226 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312252 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312289 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312352 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312375 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312420 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312457 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312505 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312554 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312580 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312620 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312658 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.312719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312701 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312738 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312768 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312791 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312805 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312977 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313062 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313165 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313228 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313439 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.313665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313537 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.312737 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.313910 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.313719 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.315166 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.315145 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.315612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.315592 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.315917 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.315899 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.316030 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.316010 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.373736 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.373657 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.373923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.373844 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tjvz\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.373923 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.373909 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-s2hxp\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.374178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.374159 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfd6j\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\") pod \"maas-default-gateway-openshift-default-6cc458d887-w4229\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.401721 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.401693 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:51.449705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.449666 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:51.848162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.848137 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:14:51.850483 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:51.850457 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3086d9_2c33_4c24_a44d_521ff1a042f2.slice/crio-389463a9f6a5a3bf36c1915c51bd2ca428601771a6aeafe0124be021fdb7b3b4 WatchSource:0}: Error finding container 389463a9f6a5a3bf36c1915c51bd2ca428601771a6aeafe0124be021fdb7b3b4: Status 404 returned error can't find the container with id 389463a9f6a5a3bf36c1915c51bd2ca428601771a6aeafe0124be021fdb7b3b4 Nov 04 02:14:51.852093 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.852069 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:14:51.853438 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.853400 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:51.853501 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.853484 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:51.853543 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.853533 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:51.853745 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:14:51.853724 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb41225_5e16_40d9_9206_32f5835e5b0f.slice/crio-3730fc3a73652eb2055b697453e599cbd49a9fb61a26b29788f69e6a507719df WatchSource:0}: Error finding container 3730fc3a73652eb2055b697453e599cbd49a9fb61a26b29788f69e6a507719df: Status 404 returned error can't find the container with id 3730fc3a73652eb2055b697453e599cbd49a9fb61a26b29788f69e6a507719df Nov 04 02:14:51.856045 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.856020 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:51.856129 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.856084 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:51.856187 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:51.856128 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:14:52.003025 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.002100 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" event={"ID":"8a3086d9-2c33-4c24-a44d-521ff1a042f2","Type":"ContainerStarted","Data":"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b"} Nov 04 02:14:52.003025 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.002182 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" event={"ID":"8a3086d9-2c33-4c24-a44d-521ff1a042f2","Type":"ContainerStarted","Data":"389463a9f6a5a3bf36c1915c51bd2ca428601771a6aeafe0124be021fdb7b3b4"} Nov 04 02:14:52.004347 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.004294 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" event={"ID":"0eb41225-5e16-40d9-9206-32f5835e5b0f","Type":"ContainerStarted","Data":"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002"} Nov 04 02:14:52.004491 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.004367 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" event={"ID":"0eb41225-5e16-40d9-9206-32f5835e5b0f","Type":"ContainerStarted","Data":"3730fc3a73652eb2055b697453e599cbd49a9fb61a26b29788f69e6a507719df"} Nov 04 02:14:52.124954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.124850 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podStartSLOduration=1.124832059 podStartE2EDuration="1.124832059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:52.12203395 +0000 UTC m=+469.817724149" watchObservedRunningTime="2026-11-04 02:14:52.124832059 +0000 UTC m=+469.820522261" Nov 04 02:14:52.245032 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.244970 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podStartSLOduration=1.244947727 podStartE2EDuration="1.244947727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:52.242988729 +0000 UTC m=+469.938678930" watchObservedRunningTime="2026-11-04 02:14:52.244947727 +0000 UTC m=+469.940637927" Nov 04 02:14:52.410407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.404777 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:14:52.410407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.408456 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:52.410407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.408572 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:52.450201 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.450159 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:14:52.451545 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.451521 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:52.451600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:52.451576 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:53.404269 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:53.403754 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:53.404269 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:53.403852 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:53.450472 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:53.450428 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:53.450653 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:53.450481 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:53.869667 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:14:53.869631 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:14:54.403785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:54.403747 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:54.404062 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:54.404033 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:54.451046 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:54.451014 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:54.451447 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:54.451083 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:55.403357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:55.402758 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:55.403357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:55.402820 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:55.450349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:55.450303 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:55.450516 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:55.450383 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:56.403774 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:56.403736 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:56.404410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:56.404378 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:56.450992 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:56.450963 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:56.451247 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:56.451226 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:57.405744 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:57.405709 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:57.406318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:57.406289 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:57.450819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:57.450790 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:57.451031 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:57.451011 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:58.402863 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:58.402820 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:58.403018 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:58.402891 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:58.450978 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:58.450943 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:58.451356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:58.451008 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:14:59.402896 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:59.402861 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-w4229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:14:59.403072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:59.402949 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:14:59.450580 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:59.450534 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-s2hxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:14:59.450771 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:14:59.450606 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:15:00.171134 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.171086 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:00.176785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.176757 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.230492 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.229872 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:15:00.245483 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.245445 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6pn\" (UniqueName: \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.245691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.245516 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.279095 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.279058 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:15:00.279095 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.279069 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:15:00.283160 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.283126 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:00.347406 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.347323 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6pn\" (UniqueName: \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.347583 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.347434 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.348389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.348360 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.410241 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.410204 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6pn\" (UniqueName: \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\") pod \"maas-api-key-cleanup-29895975-j59wz\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.496349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.496245 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:00.498819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.498794 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:15:00.499414 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.499295 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:15:00.500231 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.500176 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:15:00.556138 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.553054 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:15:00.556138 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.553931 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:15:00.559758 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.559637 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:15:00.877866 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:00.877826 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:01.094851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:01.094809 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerStarted","Data":"7d9f8e64d5ca92cc35a622dadfd631d46bf4b6f53b75d1ba6a6b7329e0595120"} Nov 04 02:15:06.221841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.221812 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:15:06.225472 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.225451 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410541 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410508 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410541 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410545 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzhd\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410586 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410704 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410744 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410767 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410878 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410937 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410928 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.410966 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.410951 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.437559 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.437524 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:15:06.511710 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511623 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.511710 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511688 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.511942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511716 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.511942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511747 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.511942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511771 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzhd\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.511942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511924 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.511978 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512016 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512043 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512347 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512153 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512347 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512323 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512504 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512393 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512504 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512468 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.512642 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.512625 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.514158 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.514134 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.514376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.514358 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.572614 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.572585 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.572752 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.572667 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzhd\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\") pod \"maas-default-gateway-openshift-default-6cc458d887-79sc8\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:06.837741 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:06.837705 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:07.130915 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.130840 6073 generic.go:356] "Generic (PLEG): container finished" podID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerID="3eea4fe8e48cddf6b1fec5dcb03c1bfcfda4ecb95eb25836bb50fce6176f6498" exitCode=60 Nov 04 02:15:07.130915 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.130881 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerDied","Data":"3eea4fe8e48cddf6b1fec5dcb03c1bfcfda4ecb95eb25836bb50fce6176f6498"} Nov 04 02:15:07.131193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.131176 6073 scope.go:122] "RemoveContainer" containerID="3eea4fe8e48cddf6b1fec5dcb03c1bfcfda4ecb95eb25836bb50fce6176f6498" Nov 04 02:15:07.263076 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:15:07.263053 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c856dab_72cf_46ff_ba91_98bcc3161899.slice/crio-f8ecb03dfecc945466eb9369d249b325b3bfe13713f9dcdbd18915545782ba88 WatchSource:0}: Error finding container f8ecb03dfecc945466eb9369d249b325b3bfe13713f9dcdbd18915545782ba88: Status 404 returned error can't find the container with id f8ecb03dfecc945466eb9369d249b325b3bfe13713f9dcdbd18915545782ba88 Nov 04 02:15:07.265534 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.265496 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:07.265622 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.265589 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:07.265677 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.265645 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:07.283822 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:07.283797 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:15:08.139825 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.139793 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerStarted","Data":"647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5"} Nov 04 02:15:08.141506 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.141480 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" event={"ID":"4c856dab-72cf-46ff-ba91-98bcc3161899","Type":"ContainerStarted","Data":"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0"} Nov 04 02:15:08.141602 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.141533 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" event={"ID":"4c856dab-72cf-46ff-ba91-98bcc3161899","Type":"ContainerStarted","Data":"f8ecb03dfecc945466eb9369d249b325b3bfe13713f9dcdbd18915545782ba88"} Nov 04 02:15:08.260065 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.260019 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" podStartSLOduration=3.279956216 podStartE2EDuration="8.260004374s" totalImagesPullingTime="4.980048158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:08.259555326 +0000 UTC m=+485.955245526" watchObservedRunningTime="2026-11-04 02:15:08.260004374 +0000 UTC m=+485.955694572" Nov 04 02:15:08.387393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.387326 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podStartSLOduration=2.387313322 podStartE2EDuration="2.387313322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:08.385105553 +0000 UTC m=+486.080795784" watchObservedRunningTime="2026-11-04 02:15:08.387313322 +0000 UTC m=+486.083003521" Nov 04 02:15:08.838296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.838261 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:08.839780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.839760 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:08.839853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:08.839815 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:09.149574 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.149494 6073 generic.go:356] "Generic (PLEG): container finished" podID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerID="647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5" exitCode=60 Nov 04 02:15:09.149747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.149580 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerDied","Data":"647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5"} Nov 04 02:15:09.149747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.149658 6073 scope.go:122] "RemoveContainer" containerID="3eea4fe8e48cddf6b1fec5dcb03c1bfcfda4ecb95eb25836bb50fce6176f6498" Nov 04 02:15:09.149888 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.149866 6073 scope.go:122] "RemoveContainer" containerID="647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5" Nov 04 02:15:09.150170 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:15:09.150137 6073 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-j59wz_odh-ai-gateway-infra(a002813d-b48a-458c-b63c-09dbe62d13d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" Nov 04 02:15:09.838651 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.838620 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:09.839116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:09.838688 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:10.156606 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:10.156530 6073 scope.go:122] "RemoveContainer" containerID="647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5" Nov 04 02:15:10.156803 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:15:10.156781 6073 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-j59wz_odh-ai-gateway-infra(a002813d-b48a-458c-b63c-09dbe62d13d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" Nov 04 02:15:10.838827 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:10.838788 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:10.839269 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:10.838845 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:11.838756 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:11.838723 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:11.838942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:11.838775 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:12.842572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:12.842539 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:12.843183 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:12.843131 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:13.844357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:13.843776 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:13.844357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:13.843840 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:14.838005 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:14.837964 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 02:15:14.838196 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:14.838023 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 02:15:16.838649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:16.838606 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 02:15:16.839037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:16.838679 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 02:15:17.881455 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:17.881423 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-79sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 02:15:17.881842 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:17.881472 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 02:15:18.298256 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:18.298221 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:18.298679 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:18.298660 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:18.299675 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:18.299655 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:15:20.419389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:20.419351 6073 scope.go:122] "RemoveContainer" containerID="647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5" Nov 04 02:15:21.231858 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:21.231830 6073 generic.go:356] "Generic (PLEG): container finished" podID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerID="620a552ffa80c026ff1f3941e873098bedf6c0a00740dadd6c835fe562270bdd" exitCode=60 Nov 04 02:15:21.232003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:21.231911 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerDied","Data":"620a552ffa80c026ff1f3941e873098bedf6c0a00740dadd6c835fe562270bdd"} Nov 04 02:15:21.232003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:21.231978 6073 scope.go:122] "RemoveContainer" containerID="647b3902a707152d08e8170b90cdd95142b1faafbc3737e11022aa0e17d761b5" Nov 04 02:15:21.232393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:21.232378 6073 scope.go:122] "RemoveContainer" containerID="620a552ffa80c026ff1f3941e873098bedf6c0a00740dadd6c835fe562270bdd" Nov 04 02:15:21.232700 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:15:21.232679 6073 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-j59wz_odh-ai-gateway-infra(a002813d-b48a-458c-b63c-09dbe62d13d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" Nov 04 02:15:22.356438 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.356409 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:22.501941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.501917 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:22.579985 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.579957 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\" (UniqueName: \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\") pod \"a002813d-b48a-458c-b63c-09dbe62d13d1\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " Nov 04 02:15:22.580147 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.580022 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\") pod \"a002813d-b48a-458c-b63c-09dbe62d13d1\" (UID: \"a002813d-b48a-458c-b63c-09dbe62d13d1\") " Nov 04 02:15:22.580425 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.580402 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca" pod "a002813d-b48a-458c-b63c-09dbe62d13d1" (UID: "a002813d-b48a-458c-b63c-09dbe62d13d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:15:22.582196 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.582176 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn" pod "a002813d-b48a-458c-b63c-09dbe62d13d1" (UID: "a002813d-b48a-458c-b63c-09dbe62d13d1"). InnerVolumeSpecName "kube-api-access-fj6pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:15:22.681039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.680958 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a002813d-b48a-458c-b63c-09dbe62d13d1-maas-api-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:15:22.681039 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:22.680988 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj6pn\" (UniqueName: \"kubernetes.io/projected/a002813d-b48a-458c-b63c-09dbe62d13d1-kube-api-access-fj6pn\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:15:23.246148 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:23.246117 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" Nov 04 02:15:23.246401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:23.246148 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz" event={"ID":"a002813d-b48a-458c-b63c-09dbe62d13d1","Type":"ContainerDied","Data":"7d9f8e64d5ca92cc35a622dadfd631d46bf4b6f53b75d1ba6a6b7329e0595120"} Nov 04 02:15:23.246401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:23.246209 6073 scope.go:122] "RemoveContainer" containerID="620a552ffa80c026ff1f3941e873098bedf6c0a00740dadd6c835fe562270bdd" Nov 04 02:15:23.410265 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:23.410229 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:23.413306 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:23.413279 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-j59wz"] Nov 04 02:15:23.857546 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:15:23.857508 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:15:24.425623 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:24.425589 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" path="/var/lib/kubelet/pods/a002813d-b48a-458c-b63c-09dbe62d13d1/volumes" Nov 04 02:15:36.458884 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.458849 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2"] Nov 04 02:15:36.459294 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459281 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459295 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459311 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459315 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459326 6073 cpu_manager.go:414] "containerMap: removing container" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459346 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459519 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459410 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.459519 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.459418 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:15:36.463898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.463881 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524302 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524442 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524495 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgtb6\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-kube-api-access-qgtb6\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524518 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524570 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524596 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524771 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524750 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524806 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524787 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.524839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.524817 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.567562 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.567533 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2"] Nov 04 02:15:36.625526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625492 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625545 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625578 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgtb6\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-kube-api-access-qgtb6\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625600 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625628 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625649 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625696 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.625724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625718 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626102 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.625748 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626228 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.626190 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626290 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.626273 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.626314 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.626529 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.626637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.626618 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.628124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.628105 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.628255 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.628236 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.686828 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.686793 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.686984 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.686838 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgtb6\" (UniqueName: \"kubernetes.io/projected/98fc712a-69ab-4d51-aa0f-8feda896bc2c-kube-api-access-qgtb6\") pod \"maas-default-gateway-openshift-default-6cc458d887-dmst2\" (UID: \"98fc712a-69ab-4d51-aa0f-8feda896bc2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:36.776637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:36.776611 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:37.201564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.201538 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2"] Nov 04 02:15:37.202965 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:15:37.202936 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fc712a_69ab_4d51_aa0f_8feda896bc2c.slice/crio-638fa8d64c1081cc0958539142e80a76ad99c84ecb415cb73936fc900ccfd91a WatchSource:0}: Error finding container 638fa8d64c1081cc0958539142e80a76ad99c84ecb415cb73936fc900ccfd91a: Status 404 returned error can't find the container with id 638fa8d64c1081cc0958539142e80a76ad99c84ecb415cb73936fc900ccfd91a Nov 04 02:15:37.205185 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.205156 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:37.205252 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.205220 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:37.205252 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.205248 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:15:37.324601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.324560 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" event={"ID":"98fc712a-69ab-4d51-aa0f-8feda896bc2c","Type":"ContainerStarted","Data":"10c6129516fd57d877d182c9c883b83cc1a6f8fe4f78177bc432ff631dcc45af"} Nov 04 02:15:37.324762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.324630 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" event={"ID":"98fc712a-69ab-4d51-aa0f-8feda896bc2c","Type":"ContainerStarted","Data":"638fa8d64c1081cc0958539142e80a76ad99c84ecb415cb73936fc900ccfd91a"} Nov 04 02:15:37.446500 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.446443 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podStartSLOduration=1.44642735 podStartE2EDuration="1.44642735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:37.44425678 +0000 UTC m=+515.139946996" watchObservedRunningTime="2026-11-04 02:15:37.44642735 +0000 UTC m=+515.142117549" Nov 04 02:15:37.776717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:37.776687 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:38.778603 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:38.778568 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:38.778990 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:38.778626 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:39.777055 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:39.777023 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:39.777280 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:39.777087 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:40.777424 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:40.777374 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:40.777853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:40.777447 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:41.777672 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:41.777640 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:41.778078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:41.777694 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:42.777684 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:42.777648 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:42.778078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:42.777706 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:43.777841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:43.777795 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:43.778235 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:43.777870 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:44.777937 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:44.777901 6073 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-dmst2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 02:15:44.778370 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:44.777956 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" podUID="98fc712a-69ab-4d51-aa0f-8feda896bc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 02:15:45.796381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:45.796318 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:45.797159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:45.797135 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:45.797990 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:15:45.797969 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-dmst2" Nov 04 02:15:53.857524 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:15:53.857484 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:16:06.849098 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.849062 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:16:06.849722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.849686 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="a002813d-b48a-458c-b63c-09dbe62d13d1" containerName="cleanup" Nov 04 02:16:06.854175 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.854153 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.906995 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.906963 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-5a27ba-openshift-default-dockercfg-r2gf6\"" Nov 04 02:16:06.958907 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.958877 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:16:06.973521 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973486 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973521 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973524 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973777 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973556 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973777 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973692 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973777 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973747 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5zz\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973899 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973795 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973899 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973889 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973927 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:06.973979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:06.973949 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.074700 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074662 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.074883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074713 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5zz\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.074883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074742 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.074883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074777 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.074883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074806 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.074941 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075043 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075088 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075132 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075305 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075250 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075430 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075406 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075513 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075478 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075580 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.075646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.075609 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.077496 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.077474 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.077585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.077540 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.135870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.135799 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.135870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.135855 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5zz\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.168830 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.168802 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:07.216398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.216363 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:16:07.220407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.220382 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.284210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.284125 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:16:07.359199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.359168 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-dd5d18-openshift-default-dockercfg-5qdvr\"" Nov 04 02:16:07.378928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.378893 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htl8n\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379103 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.378944 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379103 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379085 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379136 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379262 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379245 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379299 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379278 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379299 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379296 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379314 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.379399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.379380 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480141 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480054 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480141 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480118 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htl8n\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480146 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480179 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480219 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480270 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480301 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480324 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.480403 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480390 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.481007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.480981 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.481161 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.481145 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.481239 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.481186 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.481239 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.481204 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.481368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.481271 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.484011 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.483995 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.484258 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.484239 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.536147 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.536117 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:16:07.541507 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.541483 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.545169 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.545143 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.545325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.545274 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htl8n\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.599955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.599928 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-e34b54-openshift-default-dockercfg-vpxtz\"" Nov 04 02:16:07.604083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.604038 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:16:07.624241 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.624218 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:16:07.627223 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:07.627197 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1d2cf9_6cb5_47a0_b3f8_34f5d0ceaf94.slice/crio-18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0 WatchSource:0}: Error finding container 18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0: Status 404 returned error can't find the container with id 18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0 Nov 04 02:16:07.630118 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.630085 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:07.630211 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.630152 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:07.630211 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.630179 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:07.682863 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.682826 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683030 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.682869 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683030 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.682999 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683033 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683054 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lsbb\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683274 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683193 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683274 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683249 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683348 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.683446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.683401 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784370 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784319 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784400 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784456 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784511 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784540 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784591 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784613 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784643 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lsbb\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.784819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.784693 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.785113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.785079 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.785243 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.785170 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.785314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.785245 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.785314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.785276 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.785435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.785402 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.787056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.787035 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.787265 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.787248 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.844091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.844046 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:07.846879 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.846851 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.847017 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.846916 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lsbb\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.854534 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.854506 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:07.866035 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.866007 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:16:07.870740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.870715 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:07.931268 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.931042 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-8cf267-openshift-default-dockercfg-7jsf4\"" Nov 04 02:16:07.935671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:07.934603 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022099 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022261 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022303 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022353 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022381 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022447 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022509 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022552 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrk6\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.023409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.022595 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123305 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123229 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123321 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123384 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123415 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123482 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123437 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123500 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123563 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123608 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrk6\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123763 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123809 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.123994 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.123970 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.124388 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.124367 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.124535 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.124515 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.124615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.124597 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.125986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.125967 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.125986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.125980 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.186289 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.186258 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrk6\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.186520 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.186494 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.188159 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.188139 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:16:08.193067 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.193043 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.221129 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.221097 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:08.246720 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.246697 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-9adedd-openshift-default-dockercfg-4x2k2\"" Nov 04 02:16:08.247150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.246325 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:16:08.248291 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:08.248265 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e94ae4_8c5a_472b_b107_48e929c3caa2.slice/crio-91952b225ccdb29daf638be18893ca860cfb9f494b04b08853a2a474aba6d0f6 WatchSource:0}: Error finding container 91952b225ccdb29daf638be18893ca860cfb9f494b04b08853a2a474aba6d0f6: Status 404 returned error can't find the container with id 91952b225ccdb29daf638be18893ca860cfb9f494b04b08853a2a474aba6d0f6 Nov 04 02:16:08.253724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.251488 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.253724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.251580 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.253724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.251626 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.254679 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.254643 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:16:08.300072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.299985 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:16:08.304526 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:08.304480 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb784a230_c6a9_4829_a740_3be96fd836f5.slice/crio-2fa93a65859430b5eea6e1334358c7696461091d62af144b040010d54577b474 WatchSource:0}: Error finding container 2fa93a65859430b5eea6e1334358c7696461091d62af144b040010d54577b474: Status 404 returned error can't find the container with id 2fa93a65859430b5eea6e1334358c7696461091d62af144b040010d54577b474 Nov 04 02:16:08.307311 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.307235 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.307458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.307383 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.307458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.307431 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328615 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328678 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328811 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328876 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328902 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.329124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.328943 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.332986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.330165 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njk4j\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.332986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.330246 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.332986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.330284 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.434924 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.434881 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njk4j\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.434975 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435015 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435054 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435082 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435218 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435258 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435282 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435324 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.435820 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.435794 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.436449 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.436418 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.437610 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.437586 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.437859 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.437840 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.438517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.438497 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.439568 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.439548 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.440040 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.440020 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.499550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.499520 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njk4j\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.499724 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.499638 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.506410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.506382 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:16:08.511672 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.511640 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.517008 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.516978 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:08.535515 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.535485 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerStarted","Data":"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea"} Nov 04 02:16:08.535641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.535543 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerStarted","Data":"18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0"} Nov 04 02:16:08.574422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.570846 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerStarted","Data":"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0"} Nov 04 02:16:08.574422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.570916 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerStarted","Data":"2fa93a65859430b5eea6e1334358c7696461091d62af144b040010d54577b474"} Nov 04 02:16:08.574422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.571872 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e970c7-openshift-default-dockercfg-rlnm2\"" Nov 04 02:16:08.575063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.575026 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:16:08.580948 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.580093 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerStarted","Data":"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24"} Nov 04 02:16:08.580948 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.580152 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerStarted","Data":"91952b225ccdb29daf638be18893ca860cfb9f494b04b08853a2a474aba6d0f6"} Nov 04 02:16:08.616906 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.616874 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:16:08.617735 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:08.617691 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfaacc0_62da_4634_a8c6_4e38f86ffe09.slice/crio-465bd45e82100dbae0f3a37961f939c1081d99143136ea7277d248ba0e4029bb WatchSource:0}: Error finding container 465bd45e82100dbae0f3a37961f939c1081d99143136ea7277d248ba0e4029bb: Status 404 returned error can't find the container with id 465bd45e82100dbae0f3a37961f939c1081d99143136ea7277d248ba0e4029bb Nov 04 02:16:08.621726 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.620987 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.621726 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.621076 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.621726 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.621112 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639588 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639637 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639718 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639815 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639845 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx8jz\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639906 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.639964 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.640012 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.641354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.640072 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741398 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741440 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx8jz\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741498 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741559 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741626 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741683 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741914 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741826 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741914 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741866 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741914 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741920 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.741914 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.741865 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.742168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.742054 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.742852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.742416 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.742852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.742547 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.742852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.742808 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.745273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.745253 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.745413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.745288 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.761322 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.761269 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podStartSLOduration=2.761254181 podStartE2EDuration="2.761254181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:08.758053581 +0000 UTC m=+546.453743783" watchObservedRunningTime="2026-11-04 02:16:08.761254181 +0000 UTC m=+546.456944380" Nov 04 02:16:08.802823 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.802790 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.802980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.802790 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx8jz\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\") pod \"e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.835569 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.835535 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:08.844729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.844691 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:08.855438 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.855409 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:08.891168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.891097 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podStartSLOduration=1.891072468 podStartE2EDuration="1.891072468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:08.883777978 +0000 UTC m=+546.579468178" watchObservedRunningTime="2026-11-04 02:16:08.891072468 +0000 UTC m=+546.586762668" Nov 04 02:16:08.947218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.947166 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.949944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.947320 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.949944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.947424 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:08.949944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:08.949934 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:16:09.020056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.019996 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podStartSLOduration=2.019976675 podStartE2EDuration="2.019976675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:09.011662181 +0000 UTC m=+546.707352383" watchObservedRunningTime="2026-11-04 02:16:09.019976675 +0000 UTC m=+546.715666874" Nov 04 02:16:09.138407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.138371 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:16:09.143028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.143005 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.172409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.169624 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:09.187388 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.184260 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:09.197396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.197370 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-9b0f0d-openshift-default-dockercfg-4pmml\"" Nov 04 02:16:09.202870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.202843 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:16:09.239150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.239115 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:16:09.241509 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:09.241477 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e50be5_0c8e_45ec_8b8f_ba696163f1cf.slice/crio-d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5 WatchSource:0}: Error finding container d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5: Status 404 returned error can't find the container with id d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5 Nov 04 02:16:09.244292 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.244254 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:09.244418 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.244398 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:09.244481 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.244465 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:09.254258 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254218 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254279 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254392 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254489 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254536 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254561 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254689 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254671 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254794 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254770 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.254949 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.254929 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs529\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356590 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356505 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs529\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356593 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356627 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356656 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356697 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.356740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356735 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356762 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356854 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.356897 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.357080 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357452 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.357425 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357750 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.357463 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357750 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.357677 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.357874 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.357790 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.359345 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.359303 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.359678 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.359660 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.417787 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.417752 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs529\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.417955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.417768 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\") pod \"e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.465089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.465010 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:09.589603 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.589565 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerStarted","Data":"668f60e7790dff9dace0f733a20d7bde3a0a53ee4a2174352e0fe97efce23928"} Nov 04 02:16:09.589794 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.589629 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerStarted","Data":"590841a18380819e68ecb2ea78ea4ec3a43104e70a851994b05f7a107c3ecedf"} Nov 04 02:16:09.591594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.591559 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerStarted","Data":"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e"} Nov 04 02:16:09.591725 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.591620 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerStarted","Data":"465bd45e82100dbae0f3a37961f939c1081d99143136ea7277d248ba0e4029bb"} Nov 04 02:16:09.593379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.593319 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerStarted","Data":"491b286cfa5933f86b8518dd0910bf4aca9acf4f9b63f7e8ac4f0cc61fbea8c7"} Nov 04 02:16:09.593505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.593408 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerStarted","Data":"d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5"} Nov 04 02:16:09.593733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.593714 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:09.594669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.594651 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:16:09.712706 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.712649 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podStartSLOduration=1.712634387 podStartE2EDuration="1.712634387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:09.712104577 +0000 UTC m=+547.407794777" watchObservedRunningTime="2026-11-04 02:16:09.712634387 +0000 UTC m=+547.408324586" Nov 04 02:16:09.846356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.837638 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:09.849318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.849260 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podStartSLOduration=2.84924304 podStartE2EDuration="2.84924304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:09.838123367 +0000 UTC m=+547.533813569" watchObservedRunningTime="2026-11-04 02:16:09.84924304 +0000 UTC m=+547.544933313" Nov 04 02:16:09.856513 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.856484 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:16:09.857682 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.857660 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:09.858027 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:09.858003 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd2903b_2f65_4e69_a3e3_09a63082cbc1.slice/crio-8df4e5e7c7a92dba78b493ee2d7f021f02c613737d2650ef8dacd0495f829696 WatchSource:0}: Error finding container 8df4e5e7c7a92dba78b493ee2d7f021f02c613737d2650ef8dacd0495f829696: Status 404 returned error can't find the container with id 8df4e5e7c7a92dba78b493ee2d7f021f02c613737d2650ef8dacd0495f829696 Nov 04 02:16:09.860238 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.860217 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:09.860667 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.860640 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:09.860713 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.860705 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:09.860744 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:09.860734 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:10.086634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.086577 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podStartSLOduration=2.086560247 podStartE2EDuration="2.086560247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:10.085281174 +0000 UTC m=+547.780971374" watchObservedRunningTime="2026-11-04 02:16:10.086560247 +0000 UTC m=+547.782250446" Nov 04 02:16:10.224581 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.224490 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:10.230808 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.230780 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:10.517130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.517100 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:10.522589 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.522561 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:10.601637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.601596 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerStarted","Data":"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d"} Nov 04 02:16:10.601844 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.601656 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerStarted","Data":"8df4e5e7c7a92dba78b493ee2d7f021f02c613737d2650ef8dacd0495f829696"} Nov 04 02:16:10.602547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602512 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:10.602765 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602740 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:10.602893 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602781 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:10.602893 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602805 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:10.602893 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602858 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:10.603113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.602953 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:10.603369 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.603350 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:10.603435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.603402 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:16:10.841759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:10.841685 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:11.037968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.037916 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podStartSLOduration=2.037900052 podStartE2EDuration="2.037900052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:11.036276872 +0000 UTC m=+548.731967073" watchObservedRunningTime="2026-11-04 02:16:11.037900052 +0000 UTC m=+548.733590251" Nov 04 02:16:11.465697 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.465663 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:11.471190 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.471166 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:11.608688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.608660 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:11.608849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.608708 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:11.608849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.608749 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:16:11.608849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:11.608781 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:16:23.879174 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:16:23.878998 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:16:37.782076 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.782035 6073 generic.go:356] "Generic (PLEG): container finished" podID="b784a230-c6a9-4829-a740-3be96fd836f5" containerID="168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0" exitCode=0 Nov 04 02:16:37.782640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.782099 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerDied","Data":"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0"} Nov 04 02:16:37.782640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.782515 6073 scope.go:122] "RemoveContainer" containerID="168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0" Nov 04 02:16:37.786115 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.786079 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:37.786218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.786151 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:37.786218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:37.786183 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:38.789721 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.789682 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerStarted","Data":"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40"} Nov 04 02:16:38.828320 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.828281 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:16:38.848768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.848734 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:38.870934 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.870906 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:38.873742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.873711 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:38.875439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.873999 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:38.885987 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.885961 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.968034 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.968128 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.968217 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p956t\"" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.968269 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-8cf267-serving-cert\"" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.968368 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.982514 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.982592 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.982683 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7mf4\" (UniqueName: \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:38.983353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:38.982731 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.083401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.083296 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7mf4\" (UniqueName: \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.083401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.083388 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.083643 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.083476 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.083643 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.083523 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.084206 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.084176 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.087002 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.086978 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.090307 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.090283 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.179462 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.175090 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7mf4\" (UniqueName: \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\") pod \"maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.189236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.189212 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:39.985124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.985086 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:16:39.985819 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:39.985101 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af1abc7_9563_46e8_9a54_138c336d4563.slice/crio-66abd7c603ff5c7cb40557e421f36fca8217b51616b35622bc0541443d120972 WatchSource:0}: Error finding container 66abd7c603ff5c7cb40557e421f36fca8217b51616b35622bc0541443d120972: Status 404 returned error can't find the container with id 66abd7c603ff5c7cb40557e421f36fca8217b51616b35622bc0541443d120972 Nov 04 02:16:39.986367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.986307 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:39.986570 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:39.986542 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:40.607064 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:40.607028 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:40.855762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:40.855725 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:40.856106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:40.856072 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:41.053605 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:41.053569 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" event={"ID":"4af1abc7-9563-46e8-9a54-138c336d4563","Type":"ContainerStarted","Data":"66abd7c603ff5c7cb40557e421f36fca8217b51616b35622bc0541443d120972"} Nov 04 02:16:41.856357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:41.855834 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:41.856357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:41.855899 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:42.095361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:42.092018 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" event={"ID":"4af1abc7-9563-46e8-9a54-138c336d4563","Type":"ContainerStarted","Data":"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce"} Nov 04 02:16:42.099357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:42.096547 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:42.272478 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:42.249067 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" podStartSLOduration=3.244314504 podStartE2EDuration="4.249046309s" totalImagesPullingTime="1.004731805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:42.245817385 +0000 UTC m=+579.941507588" watchObservedRunningTime="2026-11-04 02:16:42.249046309 +0000 UTC m=+579.944736509" Nov 04 02:16:42.858075 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:42.857977 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:42.858075 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:42.858060 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:43.855870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:43.855832 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 02:16:43.856392 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:43.855888 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 02:16:44.860897 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:44.860865 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:44.862164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:44.862144 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:16:47.156454 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.156418 6073 generic.go:356] "Generic (PLEG): container finished" podID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerID="668f60e7790dff9dace0f733a20d7bde3a0a53ee4a2174352e0fe97efce23928" exitCode=0 Nov 04 02:16:47.156928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.156486 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerDied","Data":"668f60e7790dff9dace0f733a20d7bde3a0a53ee4a2174352e0fe97efce23928"} Nov 04 02:16:47.156982 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.156956 6073 scope.go:122] "RemoveContainer" containerID="668f60e7790dff9dace0f733a20d7bde3a0a53ee4a2174352e0fe97efce23928" Nov 04 02:16:47.158417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158394 6073 generic.go:356] "Generic (PLEG): container finished" podID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerID="5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24" exitCode=0 Nov 04 02:16:47.158749 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158471 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerDied","Data":"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24"} Nov 04 02:16:47.158749 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158519 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:47.158749 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158588 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:47.158749 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158627 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:47.158892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.158788 6073 scope.go:122] "RemoveContainer" containerID="5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24" Nov 04 02:16:47.159993 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.159969 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:47.160059 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.160028 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:47.160097 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:47.160059 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:48.156358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.155884 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:16:48.205399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.199642 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerStarted","Data":"e91355bda496d71200feab5b9663ee71bc833ff8af36f1d69be828ba91eb969a"} Nov 04 02:16:48.222358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.221890 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerStarted","Data":"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a"} Nov 04 02:16:48.543927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.543896 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:48.544195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.544006 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:48.544366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.544319 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:48.873816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.873729 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:48.898477 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.897861 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:48.898748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:48.898722 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:49.523910 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:49.523874 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:49.524582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:49.524527 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:49.853417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:49.846218 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:49.853417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:49.846281 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:50.524582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:50.524547 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:50.525172 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:50.525137 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:50.845415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:50.844874 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:50.845415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:50.844957 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:51.530358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:51.517712 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:51.530358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:51.517774 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:51.875417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:51.861783 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:51.875417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:51.861877 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:52.526288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:52.525792 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:52.526288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:52.525871 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:52.856422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:52.856170 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:52.856422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:52.856233 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:53.519128 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.519086 6073 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 02:16:53.519467 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.519433 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 02:16:53.685399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.685319 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:16:53.691004 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.690975 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:53.745195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.745161 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:16:53.754455 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.751298 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-e970c7-serving-cert\"" Nov 04 02:16:53.884452 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.876071 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 02:16:53.884452 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.876131 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 02:16:53.935395 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.891177 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:53.935395 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.891364 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:53.935395 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.891441 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:53.935395 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:53.891596 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpm9b\" (UniqueName: \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.041376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.020281 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.041376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.020386 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.041376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.032450 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.041376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.032670 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpm9b\" (UniqueName: \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.041376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.033992 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.053770 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.053700 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.140405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.128002 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpm9b\" (UniqueName: \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.140405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.128775 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\") pod \"maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.249608 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:16:54.249576 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:16:54.418375 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.417944 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:54.552243 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.542017 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:54.552243 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.543097 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:54.552243 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.549563 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:16:54.850556 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.850033 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:16:54.850947 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.850922 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:54.851421 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.851389 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:54.852296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:54.852274 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:16:54.852735 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:16:54.852697 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9602d0ea_18a5_4973_a680_5834573e5959.slice/crio-51228fb1ed1b10458bddde108c0536d5a401bedd198a644829bb941e4c97e227 WatchSource:0}: Error finding container 51228fb1ed1b10458bddde108c0536d5a401bedd198a644829bb941e4c97e227: Status 404 returned error can't find the container with id 51228fb1ed1b10458bddde108c0536d5a401bedd198a644829bb941e4c97e227 Nov 04 02:16:55.485062 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:55.485020 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" event={"ID":"9602d0ea-18a5-4973-a680-5834573e5959","Type":"ContainerStarted","Data":"51228fb1ed1b10458bddde108c0536d5a401bedd198a644829bb941e4c97e227"} Nov 04 02:16:56.493539 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:56.493486 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" event={"ID":"9602d0ea-18a5-4973-a680-5834573e5959","Type":"ContainerStarted","Data":"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3"} Nov 04 02:16:56.494058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:56.493587 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:16:56.611839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:56.611776 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" podStartSLOduration=2.766670637 podStartE2EDuration="3.611755388s" totalImagesPullingTime="845.084751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:56.610645808 +0000 UTC m=+594.306336010" watchObservedRunningTime="2026-11-04 02:16:56.611755388 +0000 UTC m=+594.307445589" Nov 04 02:16:56.643493 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:56.643449 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 02:16:56.643665 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:56.643516 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 02:16:57.649587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:57.649542 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 02:16:57.650017 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:57.649623 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 02:16:58.507968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.507880 6073 generic.go:356] "Generic (PLEG): container finished" podID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerID="99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d" exitCode=0 Nov 04 02:16:58.508162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.507962 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerDied","Data":"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d"} Nov 04 02:16:58.508467 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.508449 6073 scope.go:122] "RemoveContainer" containerID="99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d" Nov 04 02:16:58.509849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.509830 6073 generic.go:356] "Generic (PLEG): container finished" podID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerID="1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e" exitCode=0 Nov 04 02:16:58.509929 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.509904 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerDied","Data":"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e"} Nov 04 02:16:58.509995 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.509970 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:58.510050 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.510038 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:58.510084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.510072 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:58.510287 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.510271 6073 scope.go:122] "RemoveContainer" containerID="1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e" Nov 04 02:16:58.511532 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.511509 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:58.511615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.511587 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:58.511656 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:58.511618 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:16:59.560360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:59.554758 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerStarted","Data":"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3"} Nov 04 02:16:59.560360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:16:59.558557 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerStarted","Data":"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb"} Nov 04 02:17:00.226356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.225794 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:17:00.226356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.225934 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 02:17:00.226356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.225978 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 02:17:00.470487 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.466786 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:17:00.470487 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.466927 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 02:17:00.470487 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:00.466974 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 02:17:01.222358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:01.221882 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 02:17:01.222358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:01.221946 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 02:17:01.466356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:01.465779 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 02:17:01.466356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:01.465842 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 02:17:02.229356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:02.226859 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 02:17:02.229356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:02.226926 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 02:17:02.473360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:02.472937 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 02:17:02.473360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:02.473006 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 02:17:02.576963 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:02.576921 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:17:03.222360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:03.221907 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 02:17:03.222360 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:03.221987 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 02:17:03.467362 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:03.466808 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 02:17:03.467362 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:03.466875 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 02:17:04.223786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:04.223745 6073 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 02:17:04.224138 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:04.224097 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 02:17:04.468359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:04.467288 6073 patch_prober.go:29] interesting pod/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 02:17:04.468359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:04.467370 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 02:17:05.225552 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.225518 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:17:05.225976 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.225951 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:17:05.226615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.226597 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:17:05.470782 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.470709 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:17:05.471214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.471145 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:17:05.471963 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:05.471941 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:17:09.616750 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.616713 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:09.617219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.616770 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:09.684149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.684117 6073 generic.go:356] "Generic (PLEG): container finished" podID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerID="cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea" exitCode=0 Nov 04 02:17:09.684353 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.684191 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerDied","Data":"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea"} Nov 04 02:17:09.684673 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.684655 6073 scope.go:122] "RemoveContainer" containerID="cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea" Nov 04 02:17:09.686014 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.685989 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:09.686069 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.686059 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:09.686105 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:09.686091 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:10.700355 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:10.695882 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerStarted","Data":"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b"} Nov 04 02:17:11.169978 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:11.169946 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:17:11.170263 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:11.170238 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:11.170458 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:11.170431 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:12.175547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:12.174636 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:12.175547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:12.174701 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:13.170034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:13.169994 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:13.170232 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:13.170051 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:14.169978 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:14.169897 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:14.170383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:14.169982 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:15.169688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:15.169653 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:17:15.169878 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:15.169713 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:17:16.174185 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:16.174155 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:17:16.174662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:16.174581 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:17:16.175446 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:16.175428 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:17:17.817894 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.817857 6073 generic.go:356] "Generic (PLEG): container finished" podID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerID="491b286cfa5933f86b8518dd0910bf4aca9acf4f9b63f7e8ac4f0cc61fbea8c7" exitCode=0 Nov 04 02:17:17.818461 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.817926 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerDied","Data":"491b286cfa5933f86b8518dd0910bf4aca9acf4f9b63f7e8ac4f0cc61fbea8c7"} Nov 04 02:17:17.818550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.818534 6073 scope.go:122] "RemoveContainer" containerID="491b286cfa5933f86b8518dd0910bf4aca9acf4f9b63f7e8ac4f0cc61fbea8c7" Nov 04 02:17:17.819785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.819760 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:17.819873 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.819823 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:17.819873 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:17.819854 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:17:18.830390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:18.830309 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerStarted","Data":"7f4ff8ca836024b2da144766a561869e4a4cb59c5fd17747cf4ee6f33214cce1"} Nov 04 02:17:18.835832 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:18.835811 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:17:18.836008 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:18.835988 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:18.836058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:18.836042 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:19.836779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:19.836737 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:19.837265 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:19.836815 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:20.836425 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:20.836384 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:20.836612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:20.836448 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:21.836735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:21.836701 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:21.837181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:21.836759 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:22.835903 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:22.835864 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:22.836090 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:22.835925 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:23.836590 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:23.836548 6073 patch_prober.go:29] interesting pod/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 02:17:23.837096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:23.836608 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 02:17:23.859385 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:17:23.859346 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:17:24.840781 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:24.840753 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:17:24.841221 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:24.841204 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:17:24.841851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:17:24.841837 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:17:53.859362 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:17:53.859304 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:18:14.099410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.099375 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:18:14.110816 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.110778 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143215 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143170 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143427 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143235 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143427 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143398 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143579 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143448 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143579 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143505 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.143579 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.143556 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzzqj\" (UniqueName: \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.162223 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.162186 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:18:14.207886 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.207850 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e34b54\"/\"kube-root-ca.crt\"" Nov 04 02:18:14.207886 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.207850 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e34b54\"/\"openshift-service-ca.crt\"" Nov 04 02:18:14.212273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.212250 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e34b54\"/\"default-dockercfg-8gsrg\"" Nov 04 02:18:14.212400 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.212353 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e34b54\"/\"premium-w0-e34b54-kserve-self-signed-certs\"" Nov 04 02:18:14.252652 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.252200 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.252652 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.252264 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.252927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.252691 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzzqj\" (UniqueName: \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.252985 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.252939 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.253038 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.253008 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.253113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.253082 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.253919 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.253898 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.254437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.254416 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.261110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.261082 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.263651 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.263609 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.266200 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.266175 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.315249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.315218 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzzqj\" (UniqueName: \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\") pod \"premium-w0-e34b54-kserve-5f949456d7-2h66h\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.423325 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.423226 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:14.835760 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:18:14.835732 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0db11310_75d3_4c21_a248_691b25092042.slice/crio-22b7a6c5e0a939aa5deed8ccd379883b94c3aa08d7b91754f3744c28646436f5 WatchSource:0}: Error finding container 22b7a6c5e0a939aa5deed8ccd379883b94c3aa08d7b91754f3744c28646436f5: Status 404 returned error can't find the container with id 22b7a6c5e0a939aa5deed8ccd379883b94c3aa08d7b91754f3744c28646436f5 Nov 04 02:18:14.836297 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:14.836276 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:18:15.177527 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:15.177437 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" event={"ID":"0db11310-75d3-4c21-a248-691b25092042","Type":"ContainerStarted","Data":"b214212f153ca5e97a42a688af0f3b61c081aff06ebee4a126b632c09a8d988f"} Nov 04 02:18:15.177527 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:15.177480 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" event={"ID":"0db11310-75d3-4c21-a248-691b25092042","Type":"ContainerStarted","Data":"22b7a6c5e0a939aa5deed8ccd379883b94c3aa08d7b91754f3744c28646436f5"} Nov 04 02:18:15.295874 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:15.295817 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" podStartSLOduration=1.295801731 podStartE2EDuration="1.295801731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:15.293713505 +0000 UTC m=+672.989403705" watchObservedRunningTime="2026-11-04 02:18:15.295801731 +0000 UTC m=+672.991491931" Nov 04 02:18:15.425350 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:15.425293 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:23.858832 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:18:23.858789 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:18:24.430911 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:24.430005 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:24.441040 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:24.441011 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:25.248711 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:25.248684 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:18:52.840927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.840868 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:18:52.845756 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.845731 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850254 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850289 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850397 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9kpb\" (UniqueName: \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850469 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850528 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.850601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.850553 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.948503 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.948456 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"kube-root-ca.crt\"" Nov 04 02:18:52.948723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.948536 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"openshift-service-ca.crt\"" Nov 04 02:18:52.948723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.948557 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"premium-w5-dd5d18-kserve-self-signed-certs\"" Nov 04 02:18:52.948723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.948574 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"default-dockercfg-gszng\"" Nov 04 02:18:52.951150 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.951113 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:18:52.951868 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.951844 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.951979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.951877 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.951979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.951915 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9kpb\" (UniqueName: \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.951979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.951948 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.952383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.952023 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.952383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.952055 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.952861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.952795 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.953295 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.953250 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.953751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.953733 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.972746 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.957690 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:52.988708 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:52.988672 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:53.013301 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:53.013262 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9kpb\" (UniqueName: \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\") pod \"premium-w5-dd5d18-kserve-6b5c98679c-fn55g\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:53.162841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:53.162746 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:18:53.577927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:53.577901 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:18:53.578761 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:18:53.578737 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7288b257_9d95_469e_be28_b9b73bd81dde.slice/crio-683845a7513c84eb27020ee70d216a4fca1b5ba8d3bcd21d515c5ecf7ac14348 WatchSource:0}: Error finding container 683845a7513c84eb27020ee70d216a4fca1b5ba8d3bcd21d515c5ecf7ac14348: Status 404 returned error can't find the container with id 683845a7513c84eb27020ee70d216a4fca1b5ba8d3bcd21d515c5ecf7ac14348 Nov 04 02:18:53.859485 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:18:53.859404 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:18:54.427110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:54.427077 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" event={"ID":"7288b257-9d95-469e-be28-b9b73bd81dde","Type":"ContainerStarted","Data":"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d"} Nov 04 02:18:54.427293 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:54.427124 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" event={"ID":"7288b257-9d95-469e-be28-b9b73bd81dde","Type":"ContainerStarted","Data":"683845a7513c84eb27020ee70d216a4fca1b5ba8d3bcd21d515c5ecf7ac14348"} Nov 04 02:18:54.540967 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:54.540912 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" podStartSLOduration=2.5408985790000003 podStartE2EDuration="2.540898579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:54.538575229 +0000 UTC m=+712.234265430" watchObservedRunningTime="2026-11-04 02:18:54.540898579 +0000 UTC m=+712.236588779" Nov 04 02:18:55.163133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:18:55.163092 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:19:03.163683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:03.163628 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:19:03.176569 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:03.176541 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:19:03.498888 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:03.498799 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:19:20.599278 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.599199 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:19:20.603378 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.603356 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.663415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.663375 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:19:20.707105 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.707070 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-9b0f0d\"/\"openshift-service-ca.crt\"" Nov 04 02:19:20.707105 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.707109 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-9b0f0d\"/\"kube-root-ca.crt\"" Nov 04 02:19:20.707435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.707209 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-9b0f0d\"/\"default-dockercfg-hzptq\"" Nov 04 02:19:20.707435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.707241 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-9b0f0d\"/\"premium-w2-9b0f0d-kserve-self-signed-certs\"" Nov 04 02:19:20.710026 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710009 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.710197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710182 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cbq8\" (UniqueName: \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.710255 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710244 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.710364 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710314 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.710410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710399 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.710504 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.710483 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812311 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cbq8\" (UniqueName: \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812459 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812527 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812583 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812765 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812718 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812819 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812763 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.812984 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812959 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.813056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.812997 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.813686 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.813661 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.816251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.816224 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.816553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.816532 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.875305 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.875228 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cbq8\" (UniqueName: \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\") pod \"premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:20.916642 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:20.916591 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:21.360282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:21.360255 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:19:21.361961 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:19:21.361933 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64686bc7_a247_4ee9_a28b_3f2fb11aa78a.slice/crio-aedba18e65884f6c0ed246f29b4bfb9d740647d348ed07ca518785beed02fc40 WatchSource:0}: Error finding container aedba18e65884f6c0ed246f29b4bfb9d740647d348ed07ca518785beed02fc40: Status 404 returned error can't find the container with id aedba18e65884f6c0ed246f29b4bfb9d740647d348ed07ca518785beed02fc40 Nov 04 02:19:21.609146 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:21.609047 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" event={"ID":"64686bc7-a247-4ee9-a28b-3f2fb11aa78a","Type":"ContainerStarted","Data":"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6"} Nov 04 02:19:21.609146 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:21.609099 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" event={"ID":"64686bc7-a247-4ee9-a28b-3f2fb11aa78a","Type":"ContainerStarted","Data":"aedba18e65884f6c0ed246f29b4bfb9d740647d348ed07ca518785beed02fc40"} Nov 04 02:19:21.725398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:21.725318 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" podStartSLOduration=1.725302809 podStartE2EDuration="1.725302809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:21.724266636 +0000 UTC m=+739.419956836" watchObservedRunningTime="2026-11-04 02:19:21.725302809 +0000 UTC m=+739.420993010" Nov 04 02:19:21.917753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:21.917667 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:23.859848 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:19:23.859811 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:19:30.916860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:30.916816 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:30.929588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:30.929556 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:31.679165 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:31.679133 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:19:53.859629 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:19:53.859589 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:19:53.894229 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:53.894180 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:19:53.898138 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:53.898111 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:53.950618 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:53.950583 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-5a27ba\"/\"openshift-service-ca.crt\"" Nov 04 02:19:53.999779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:53.999740 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-5a27ba\"/\"kube-root-ca.crt\"" Nov 04 02:19:53.999979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:53.999967 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-5a27ba\"/\"default-dockercfg-dk4l4\"" Nov 04 02:19:54.000168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.000152 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-5a27ba\"/\"premium-w3-5a27ba-kserve-self-signed-certs\"" Nov 04 02:19:54.002804 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.002782 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:19:54.007996 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.007976 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.008073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.008037 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.008133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.008111 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.008197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.008174 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.008307 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.008288 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6pnl\" (UniqueName: \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.008433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.008418 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.109807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109717 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.109807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109785 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110067 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109842 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110067 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109871 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110067 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109897 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110067 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.109981 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6pnl\" (UniqueName: \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110289 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.110156 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110289 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.110235 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.110289 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.110266 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.112397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.112377 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.112498 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.112454 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.173424 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.173380 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6pnl\" (UniqueName: \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\") pod \"premium-w3-5a27ba-kserve-75c4575867-blhhn\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.211202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.211150 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:19:54.626314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.626282 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:19:54.627866 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:19:54.627836 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod903a542c_58f6_484a_92f2_c943aca4cc28.slice/crio-59f7375c4dc32480785eb43c128497ca36aac0ef61f1a9ab7dc363d464efb956 WatchSource:0}: Error finding container 59f7375c4dc32480785eb43c128497ca36aac0ef61f1a9ab7dc363d464efb956: Status 404 returned error can't find the container with id 59f7375c4dc32480785eb43c128497ca36aac0ef61f1a9ab7dc363d464efb956 Nov 04 02:19:54.815584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.815045 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" event={"ID":"903a542c-58f6-484a-92f2-c943aca4cc28","Type":"ContainerStarted","Data":"bf8e25ff971b4717b0b5127d25ae35cf9cd9757fa22f9b37b1dd36733418aff6"} Nov 04 02:19:54.815584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.815104 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" event={"ID":"903a542c-58f6-484a-92f2-c943aca4cc28","Type":"ContainerStarted","Data":"59f7375c4dc32480785eb43c128497ca36aac0ef61f1a9ab7dc363d464efb956"} Nov 04 02:19:54.933262 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:54.933158 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" podStartSLOduration=1.933143394 podStartE2EDuration="1.933143394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:54.932638577 +0000 UTC m=+772.628328778" watchObservedRunningTime="2026-11-04 02:19:54.933143394 +0000 UTC m=+772.628833594" Nov 04 02:19:55.211561 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:19:55.211477 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:20:04.211713 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:04.211671 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:20:04.224985 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:04.224953 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:20:04.889316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:04.889286 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:20:23.859132 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:20:23.859094 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:20:37.720457 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:37.720424 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:20:37.720979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:37.720629 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" containerID="cri-o://006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0" gracePeriod=30 Nov 04 02:20:37.770195 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:37.770162 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:20:37.770467 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:37.770428 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" containerID="cri-o://5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b" gracePeriod=30 Nov 04 02:20:43.047830 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.047793 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:20:43.061464 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.061438 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:20:43.098547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098509 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.098733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098571 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.098733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098600 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.098733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098636 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.098733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098680 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.098733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098725 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.099006 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098814 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.099006 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098828 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.099006 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.098964 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.099162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099049 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.099162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099049 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:20:43.099162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099071 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.099162 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099095 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\") pod \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\" (UID: \"8a3086d9-2c33-4c24-a44d-521ff1a042f2\") " Nov 04 02:20:43.099412 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099326 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.099957 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099930 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.100079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099962 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.100079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099978 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.100079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.099992 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.100079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.100005 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.101212 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.101184 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:20:43.101212 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.101198 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:20:43.101370 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.101191 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "kube-api-access-jfd6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:20:43.101415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.101384 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy" pod "8a3086d9-2c33-4c24-a44d-521ff1a042f2" (UID: "8a3086d9-2c33-4c24-a44d-521ff1a042f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.144433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.144395 6073 generic.go:356] "Generic (PLEG): container finished" podID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerID="006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0" exitCode=0 Nov 04 02:20:43.144593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.144457 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" Nov 04 02:20:43.144593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.144482 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" event={"ID":"4c856dab-72cf-46ff-ba91-98bcc3161899","Type":"ContainerDied","Data":"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0"} Nov 04 02:20:43.144593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.144539 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8" event={"ID":"4c856dab-72cf-46ff-ba91-98bcc3161899","Type":"ContainerDied","Data":"f8ecb03dfecc945466eb9369d249b325b3bfe13713f9dcdbd18915545782ba88"} Nov 04 02:20:43.144593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.144562 6073 scope.go:122] "RemoveContainer" containerID="006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0" Nov 04 02:20:43.145928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.145911 6073 generic.go:356] "Generic (PLEG): container finished" podID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerID="5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b" exitCode=0 Nov 04 02:20:43.146030 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.146005 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" event={"ID":"8a3086d9-2c33-4c24-a44d-521ff1a042f2","Type":"ContainerDied","Data":"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b"} Nov 04 02:20:43.146078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.145994 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" Nov 04 02:20:43.146167 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.146044 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229" event={"ID":"8a3086d9-2c33-4c24-a44d-521ff1a042f2","Type":"ContainerDied","Data":"389463a9f6a5a3bf36c1915c51bd2ca428601771a6aeafe0124be021fdb7b3b4"} Nov 04 02:20:43.155869 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.155852 6073 scope.go:122] "RemoveContainer" containerID="006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0" Nov 04 02:20:43.156137 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:20:43.156119 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0\": container with ID starting with 006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0 not found: ID does not exist" containerID="006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0" Nov 04 02:20:43.156187 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.156146 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0"} err="failed to get container status \"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0\": rpc error: code = NotFound desc = could not find container \"006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0\": container with ID starting with 006b64a43a206b20d13aacadc36534824e2c5d97fd7de4a30f2d8f34d220ccb0 not found: ID does not exist" Nov 04 02:20:43.156187 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.156164 6073 scope.go:122] "RemoveContainer" containerID="5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b" Nov 04 02:20:43.166060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.166043 6073 scope.go:122] "RemoveContainer" containerID="5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b" Nov 04 02:20:43.166353 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:20:43.166320 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b\": container with ID starting with 5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b not found: ID does not exist" containerID="5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b" Nov 04 02:20:43.166417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.166362 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b"} err="failed to get container status \"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b\": rpc error: code = NotFound desc = could not find container \"5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b\": container with ID starting with 5127bdb82fd63badd256cf791c2c4c3f05b0a7a88add075fd479853ee12c6a8b not found: ID does not exist" Nov 04 02:20:43.201264 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201230 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201264 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201268 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201312 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201385 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201450 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201469 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201500 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201532 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201585 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.201683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201615 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\") pod \"4c856dab-72cf-46ff-ba91-98bcc3161899\" (UID: \"4c856dab-72cf-46ff-ba91-98bcc3161899\") " Nov 04 02:20:43.201892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201692 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.201892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201716 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.201892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201865 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.202080 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.201992 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:20:43.202147 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202133 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202150 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202161 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c856dab-72cf-46ff-ba91-98bcc3161899-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202169 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202177 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202185 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202193 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a3086d9-2c33-4c24-a44d-521ff1a042f2-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202202 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfd6j\" (UniqueName: \"kubernetes.io/projected/8a3086d9-2c33-4c24-a44d-521ff1a042f2-kube-api-access-jfd6j\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.202431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.202212 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.203729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.203706 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:20:43.203999 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.203981 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:20:43.204203 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.204187 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "kube-api-access-5wzhd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:20:43.204261 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.204199 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy" pod "4c856dab-72cf-46ff-ba91-98bcc3161899" (UID: "4c856dab-72cf-46ff-ba91-98bcc3161899"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:20:43.304028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.303927 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.304028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.303964 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.304028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.303977 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c856dab-72cf-46ff-ba91-98bcc3161899-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.304028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.303990 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wzhd\" (UniqueName: \"kubernetes.io/projected/4c856dab-72cf-46ff-ba91-98bcc3161899-kube-api-access-5wzhd\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:43.340729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.338410 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:20:43.340729 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.338721 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-w4229"] Nov 04 02:20:43.566636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.566544 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:20:43.569838 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:43.569805 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-79sc8"] Nov 04 02:20:44.425991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:44.425962 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" path="/var/lib/kubelet/pods/4c856dab-72cf-46ff-ba91-98bcc3161899/volumes" Nov 04 02:20:44.426418 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:20:44.426405 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" path="/var/lib/kubelet/pods/8a3086d9-2c33-4c24-a44d-521ff1a042f2/volumes" Nov 04 02:20:53.858987 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:20:53.858949 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:21:09.708198 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708164 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:21:09.708748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708702 6073 cpu_manager.go:414] "containerMap: removing container" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" Nov 04 02:21:09.708748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708715 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" Nov 04 02:21:09.708748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708740 6073 cpu_manager.go:414] "containerMap: removing container" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" Nov 04 02:21:09.708748 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708745 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" Nov 04 02:21:09.708906 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708811 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c856dab-72cf-46ff-ba91-98bcc3161899" containerName="istio-proxy" Nov 04 02:21:09.708906 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.708825 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a3086d9-2c33-4c24-a44d-521ff1a042f2" containerName="istio-proxy" Nov 04 02:21:09.713437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.713417 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.817279 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.817249 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"distinct-w5-dd5d18-kserve-self-signed-certs\"" Nov 04 02:21:09.820673 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.820651 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:21:09.824696 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.824672 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.824792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.824713 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.824882 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.824861 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skbj5\" (UniqueName: \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.824921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.824912 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.825066 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.825046 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.825120 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.825103 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926198 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926168 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926198 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926209 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926300 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926320 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926419 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbj5\" (UniqueName: \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926469 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926746 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.926839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.926817 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.927091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.927069 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.928922 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.928892 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.929034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.928968 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:09.987407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:09.987299 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skbj5\" (UniqueName: \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\") pod \"distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:10.025410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:10.025375 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:10.418305 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:10.418154 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:21:10.421229 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:21:10.421196 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bf13801_e5e1_4fe5_931c_c913e086ae77.slice/crio-6ceea81796d766689c134020df01eaa8e3d15b9da815a9094661686743b96325 WatchSource:0}: Error finding container 6ceea81796d766689c134020df01eaa8e3d15b9da815a9094661686743b96325: Status 404 returned error can't find the container with id 6ceea81796d766689c134020df01eaa8e3d15b9da815a9094661686743b96325 Nov 04 02:21:11.320747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:11.320708 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" event={"ID":"2bf13801-e5e1-4fe5-931c-c913e086ae77","Type":"ContainerStarted","Data":"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792"} Nov 04 02:21:11.321157 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:11.320759 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" event={"ID":"2bf13801-e5e1-4fe5-931c-c913e086ae77","Type":"ContainerStarted","Data":"6ceea81796d766689c134020df01eaa8e3d15b9da815a9094661686743b96325"} Nov 04 02:21:11.438214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:11.438162 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" podStartSLOduration=2.438146641 podStartE2EDuration="2.438146641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:11.43636948 +0000 UTC m=+849.132059679" watchObservedRunningTime="2026-11-04 02:21:11.438146641 +0000 UTC m=+849.133836840" Nov 04 02:21:12.025772 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:12.025746 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:17.478475 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.478434 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6747-78f208"] Nov 04 02:21:17.482398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.482380 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:17.538276 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.538242 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6747-78f208"] Nov 04 02:21:17.584852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.584817 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vkvkq\"" Nov 04 02:21:17.622208 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.622179 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2btj\" (UniqueName: \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\") pod \"test-curl-6747-78f208\" (UID: \"7b5fefc7-f9b9-444b-8501-3749d4edbeba\") " pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:17.723181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.723145 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2btj\" (UniqueName: \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\") pod \"test-curl-6747-78f208\" (UID: \"7b5fefc7-f9b9-444b-8501-3749d4edbeba\") " pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:17.784835 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.784800 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2btj\" (UniqueName: \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\") pod \"test-curl-6747-78f208\" (UID: \"7b5fefc7-f9b9-444b-8501-3749d4edbeba\") " pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:17.795607 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:17.795584 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:18.171200 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:18.171172 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6747-78f208"] Nov 04 02:21:18.173104 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:21:18.173074 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b5fefc7_f9b9_444b_8501_3749d4edbeba.slice/crio-df08e9235ea28dd98c52da4b1294aa5f2ecb8ff5ffdbb62655d35b153ebe17da WatchSource:0}: Error finding container df08e9235ea28dd98c52da4b1294aa5f2ecb8ff5ffdbb62655d35b153ebe17da: Status 404 returned error can't find the container with id df08e9235ea28dd98c52da4b1294aa5f2ecb8ff5ffdbb62655d35b153ebe17da Nov 04 02:21:18.175082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:18.175067 6073 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:21:18.366204 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:18.366169 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6747-78f208" event={"ID":"7b5fefc7-f9b9-444b-8501-3749d4edbeba","Type":"ContainerStarted","Data":"df08e9235ea28dd98c52da4b1294aa5f2ecb8ff5ffdbb62655d35b153ebe17da"} Nov 04 02:21:19.373475 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:19.373439 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6747-78f208" event={"ID":"7b5fefc7-f9b9-444b-8501-3749d4edbeba","Type":"ContainerStarted","Data":"081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167"} Nov 04 02:21:19.488512 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:19.488397 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6747-78f208" podStartSLOduration=1.4408474660000001 podStartE2EDuration="2.488382898s" totalImagesPullingTime="1.047535432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:19.485500813 +0000 UTC m=+857.181191015" watchObservedRunningTime="2026-11-04 02:21:19.488382898 +0000 UTC m=+857.184073135" Nov 04 02:21:20.026454 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:20.026420 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:20.039058 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:20.039031 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:20.391157 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:20.391068 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:21:21.022417 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.022381 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6747-78f208"] Nov 04 02:21:21.027584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.027554 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6747-78f208"] Nov 04 02:21:21.188006 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.187974 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6747-849881"] Nov 04 02:21:21.188526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.188512 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" containerName="test-curl-6747-78f208" Nov 04 02:21:21.188582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.188527 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" containerName="test-curl-6747-78f208" Nov 04 02:21:21.188618 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.188610 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" containerName="test-curl-6747-78f208" Nov 04 02:21:21.192217 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.192199 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:21.246457 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.246423 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6747-849881"] Nov 04 02:21:21.269639 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.269607 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5rr\" (UniqueName: \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\") pod \"test-curl-6747-849881\" (UID: \"2a4dfb44-8f90-4154-b650-29fbe0f919f7\") " pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:21.370779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.370676 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5rr\" (UniqueName: \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\") pod \"test-curl-6747-849881\" (UID: \"2a4dfb44-8f90-4154-b650-29fbe0f919f7\") " pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:21.386459 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.386403 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6747-78f208" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" containerName="test-curl-6747-78f208" containerID="cri-o://081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167" gracePeriod=2 Nov 04 02:21:21.431655 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.431621 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5rr\" (UniqueName: \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\") pod \"test-curl-6747-849881\" (UID: \"2a4dfb44-8f90-4154-b650-29fbe0f919f7\") " pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:21.437310 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:21.437277 6073 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6747-78f208\" is forbidden: User \"system:node:ip-10-0-138-45.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-45.us-west-1.compute.internal' and this object" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:21.505000 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.504969 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:21.880391 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:21.880365 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6747-849881"] Nov 04 02:21:21.882553 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:21:21.882516 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a4dfb44_8f90_4154_b650_29fbe0f919f7.slice/crio-dd864a5ed9070b75b2c58871db2ff2e18ca4ba6246783ffe8c7306277348e4d4 WatchSource:0}: Error finding container dd864a5ed9070b75b2c58871db2ff2e18ca4ba6246783ffe8c7306277348e4d4: Status 404 returned error can't find the container with id dd864a5ed9070b75b2c58871db2ff2e18ca4ba6246783ffe8c7306277348e4d4 Nov 04 02:21:22.395650 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:22.395614 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6747-849881" event={"ID":"2a4dfb44-8f90-4154-b650-29fbe0f919f7","Type":"ContainerStarted","Data":"f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7"} Nov 04 02:21:22.395650 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:22.395656 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6747-849881" event={"ID":"2a4dfb44-8f90-4154-b650-29fbe0f919f7","Type":"ContainerStarted","Data":"dd864a5ed9070b75b2c58871db2ff2e18ca4ba6246783ffe8c7306277348e4d4"} Nov 04 02:21:22.511753 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:22.511691 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6747-849881" podStartSLOduration=1.511671674 podStartE2EDuration="1.511671674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:22.508773876 +0000 UTC m=+860.204464077" watchObservedRunningTime="2026-11-04 02:21:22.511671674 +0000 UTC m=+860.207361877" Nov 04 02:21:22.559493 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:22.559451 6073 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6747-78f208\" is forbidden: User \"system:node:ip-10-0-138-45.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-45.us-west-1.compute.internal' and this object" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:23.650134 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:23.650074 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:23.700926 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:23.700888 6073 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6747-78f208\" is forbidden: User \"system:node:ip-10-0-138-45.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-45.us-west-1.compute.internal' and this object" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:23.805691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:23.805659 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\" (UniqueName: \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\") pod \"7b5fefc7-f9b9-444b-8501-3749d4edbeba\" (UID: \"7b5fefc7-f9b9-444b-8501-3749d4edbeba\") " Nov 04 02:21:23.807965 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:23.807936 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj" pod "7b5fefc7-f9b9-444b-8501-3749d4edbeba" (UID: "7b5fefc7-f9b9-444b-8501-3749d4edbeba"). InnerVolumeSpecName "kube-api-access-q2btj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:23.859553 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:23.859518 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:21:23.907661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:23.907576 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2btj\" (UniqueName: \"kubernetes.io/projected/7b5fefc7-f9b9-444b-8501-3749d4edbeba-kube-api-access-q2btj\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:24.330048 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.330013 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6747-849881"] Nov 04 02:21:24.339028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.338992 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6747-849881"] Nov 04 02:21:24.410979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.410942 6073 generic.go:356] "Generic (PLEG): container finished" podID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" containerID="081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167" exitCode=137 Nov 04 02:21:24.411152 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.410987 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:24.411152 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.411055 6073 scope.go:122] "RemoveContainer" containerID="081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167" Nov 04 02:21:24.411315 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.411274 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6747-849881" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" containerName="test-curl-6747-849881" containerID="cri-o://f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7" gracePeriod=2 Nov 04 02:21:24.422709 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.422688 6073 scope.go:122] "RemoveContainer" containerID="081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167" Nov 04 02:21:24.423064 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:24.423036 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167\": container with ID starting with 081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167 not found: ID does not exist" containerID="081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167" Nov 04 02:21:24.423167 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.423076 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167"} err="failed to get container status \"081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167\": rpc error: code = NotFound desc = could not find container \"081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167\": container with ID starting with 081aac130bde9328ec21a6a0728f089b7ddd2e32827d60bfe61b2d6f159a3167 not found: ID does not exist" Nov 04 02:21:24.428787 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:24.428763 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" path="/var/lib/kubelet/pods/7b5fefc7-f9b9-444b-8501-3749d4edbeba/volumes" Nov 04 02:21:24.466562 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:24.466524 6073 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6747-78f208\" is forbidden: User \"system:node:ip-10-0-138-45.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-45.us-west-1.compute.internal' and this object" podUID="7b5fefc7-f9b9-444b-8501-3749d4edbeba" pod="opendatahub/test-curl-6747-78f208" Nov 04 02:21:26.589680 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:26.589654 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:26.637638 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:26.637610 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\" (UniqueName: \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\") pod \"2a4dfb44-8f90-4154-b650-29fbe0f919f7\" (UID: \"2a4dfb44-8f90-4154-b650-29fbe0f919f7\") " Nov 04 02:21:26.640190 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:26.640156 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr" pod "2a4dfb44-8f90-4154-b650-29fbe0f919f7" (UID: "2a4dfb44-8f90-4154-b650-29fbe0f919f7"). InnerVolumeSpecName "kube-api-access-6s5rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:26.738864 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:26.738782 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s5rr\" (UniqueName: \"kubernetes.io/projected/2a4dfb44-8f90-4154-b650-29fbe0f919f7-kube-api-access-6s5rr\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:27.431291 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:27.431260 6073 generic.go:356] "Generic (PLEG): container finished" podID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" containerID="f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7" exitCode=137 Nov 04 02:21:27.431502 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:27.431302 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:27.431502 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:27.431313 6073 scope.go:122] "RemoveContainer" containerID="f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7" Nov 04 02:21:27.441400 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:27.441373 6073 scope.go:122] "RemoveContainer" containerID="f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7" Nov 04 02:21:27.441729 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:27.441710 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7\": container with ID starting with f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7 not found: ID does not exist" containerID="f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7" Nov 04 02:21:27.441807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:27.441737 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7"} err="failed to get container status \"f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7\": rpc error: code = NotFound desc = could not find container \"f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7\": container with ID starting with f78db79049adf35bd5dae9ed3d8cc448a1bc6d92b0f7187552a38f43fcc043c7 not found: ID does not exist" Nov 04 02:21:27.490514 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:27.490476 6073 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6747-849881\" is forbidden: User \"system:node:ip-10-0-138-45.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-45.us-west-1.compute.internal' and this object" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" pod="opendatahub/test-curl-6747-849881" Nov 04 02:21:28.426247 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:28.426215 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" path="/var/lib/kubelet/pods/2a4dfb44-8f90-4154-b650-29fbe0f919f7/volumes" Nov 04 02:21:29.509296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:29.509260 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:21:30.906643 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.906600 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78"] Nov 04 02:21:30.907141 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.907127 6073 cpu_manager.go:414] "containerMap: removing container" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" containerName="test-curl-6747-849881" Nov 04 02:21:30.907197 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.907142 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" containerName="test-curl-6747-849881" Nov 04 02:21:30.907266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.907256 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a4dfb44-8f90-4154-b650-29fbe0f919f7" containerName="test-curl-6747-849881" Nov 04 02:21:30.912079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.912061 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:30.985954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.985922 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:30.986125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.985979 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:30.986125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:30.986108 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv6gw\" (UniqueName: \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.013669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.013630 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:21:31.013837 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.013688 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:21:31.016211 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.016192 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78"] Nov 04 02:21:31.087701 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.087667 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.087855 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.087716 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.087855 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.087803 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv6gw\" (UniqueName: \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.088518 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.088499 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.090215 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.090196 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.148927 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.148891 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv6gw\" (UniqueName: \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\") pod \"maas-api-revoke-sub-019d8c20-8xq78\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.226354 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.226261 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:31.602451 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:31.602423 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78"] Nov 04 02:21:31.604550 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:21:31.604515 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44836a2_f66f_470f_9d43_1b11e3cd82cf.slice/crio-af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c WatchSource:0}: Error finding container af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c: Status 404 returned error can't find the container with id af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c Nov 04 02:21:32.469563 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:32.469531 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" event={"ID":"d44836a2-f66f-470f-9d43-1b11e3cd82cf","Type":"ContainerStarted","Data":"ab8d45b9719ad8efe47bd1f8ab7991582d1d20e6d9776303e942e094000bae5c"} Nov 04 02:21:32.469944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:32.469580 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" event={"ID":"d44836a2-f66f-470f-9d43-1b11e3cd82cf","Type":"ContainerStarted","Data":"af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c"} Nov 04 02:21:32.585626 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:32.585574 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" podStartSLOduration=2.5855585469999998 podStartE2EDuration="2.585558547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:32.583809228 +0000 UTC m=+870.279499452" watchObservedRunningTime="2026-11-04 02:21:32.585558547 +0000 UTC m=+870.281248746" Nov 04 02:21:33.476761 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:33.476724 6073 generic.go:356] "Generic (PLEG): container finished" podID="d44836a2-f66f-470f-9d43-1b11e3cd82cf" containerID="ab8d45b9719ad8efe47bd1f8ab7991582d1d20e6d9776303e942e094000bae5c" exitCode=0 Nov 04 02:21:33.477166 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:33.476810 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" event={"ID":"d44836a2-f66f-470f-9d43-1b11e3cd82cf","Type":"ContainerDied","Data":"ab8d45b9719ad8efe47bd1f8ab7991582d1d20e6d9776303e942e094000bae5c"} Nov 04 02:21:34.645260 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.645234 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:34.732849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.732801 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\") pod \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " Nov 04 02:21:34.733055 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.732928 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\" (UniqueName: \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\") pod \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " Nov 04 02:21:34.733129 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.733076 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\") pod \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\" (UID: \"d44836a2-f66f-470f-9d43-1b11e3cd82cf\") " Nov 04 02:21:34.733726 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.733653 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca" pod "d44836a2-f66f-470f-9d43-1b11e3cd82cf" (UID: "d44836a2-f66f-470f-9d43-1b11e3cd82cf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:21:34.735953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.735919 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw" pod "d44836a2-f66f-470f-9d43-1b11e3cd82cf" (UID: "d44836a2-f66f-470f-9d43-1b11e3cd82cf"). InnerVolumeSpecName "kube-api-access-hv6gw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:34.736086 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.735996 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca" pod "d44836a2-f66f-470f-9d43-1b11e3cd82cf" (UID: "d44836a2-f66f-470f-9d43-1b11e3cd82cf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:21:34.835657 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.835621 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:34.835824 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.835663 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv6gw\" (UniqueName: \"kubernetes.io/projected/d44836a2-f66f-470f-9d43-1b11e3cd82cf-kube-api-access-hv6gw\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:34.835824 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:34.835682 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d44836a2-f66f-470f-9d43-1b11e3cd82cf-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:35.229296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:35.229266 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:21:35.491068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:35.490989 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" Nov 04 02:21:35.491068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:35.491010 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78" event={"ID":"d44836a2-f66f-470f-9d43-1b11e3cd82cf","Type":"ContainerDied","Data":"af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c"} Nov 04 02:21:35.491068 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:35.491052 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af907f2585b1a69613825a14bdb0cd3c579acfb438ec0b559c26dfedac3b1b0c" Nov 04 02:21:44.530471 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.530415 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerName="main" containerID="cri-o://cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6" gracePeriod=45 Nov 04 02:21:44.669125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.668478 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": dial tcp 10.133.0.62:8000: connect: connection refused" Nov 04 02:21:44.827209 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.827168 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:21:44.960738 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.960629 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\" (UniqueName: \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.961553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.961522 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.961688 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.961642 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\" (UniqueName: \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.961768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.961753 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.961826 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.961785 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.961883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.961824 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\") pod \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\" (UID: \"64686bc7-a247-4ee9-a28b-3f2fb11aa78a\") " Nov 04 02:21:44.962103 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.962072 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:44.962396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.962356 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:44.962532 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.962439 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:44.963401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.963357 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:44.971497 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.971475 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:44.971587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.971563 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "kube-api-access-8cbq8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:44.971587 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:44.971576 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs" pod "64686bc7-a247-4ee9-a28b-3f2fb11aa78a" (UID: "64686bc7-a247-4ee9-a28b-3f2fb11aa78a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:21:45.063592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.063509 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:45.063592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.063536 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:45.063592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.063563 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:45.063592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.063573 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cbq8\" (UniqueName: \"kubernetes.io/projected/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-kube-api-access-8cbq8\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:45.063592 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.063583 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64686bc7-a247-4ee9-a28b-3f2fb11aa78a-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:45.561163 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.561130 6073 generic.go:356] "Generic (PLEG): container finished" podID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerID="cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6" exitCode=0 Nov 04 02:21:45.561689 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.561171 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" Nov 04 02:21:45.561689 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.561192 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" event={"ID":"64686bc7-a247-4ee9-a28b-3f2fb11aa78a","Type":"ContainerDied","Data":"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6"} Nov 04 02:21:45.561689 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.561234 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55" event={"ID":"64686bc7-a247-4ee9-a28b-3f2fb11aa78a","Type":"ContainerDied","Data":"aedba18e65884f6c0ed246f29b4bfb9d740647d348ed07ca518785beed02fc40"} Nov 04 02:21:45.561689 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.561255 6073 scope.go:122] "RemoveContainer" containerID="cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6" Nov 04 02:21:45.574817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.573800 6073 scope.go:122] "RemoveContainer" containerID="cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6" Nov 04 02:21:45.574817 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:45.574633 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6\": container with ID starting with cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6 not found: ID does not exist" containerID="cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6" Nov 04 02:21:45.574817 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.574666 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6"} err="failed to get container status \"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6\": rpc error: code = NotFound desc = could not find container \"cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6\": container with ID starting with cab190ff1a6c52815bbecfabb3f85f1c47b90962f4df770d93d4f6ec40aff3f6 not found: ID does not exist" Nov 04 02:21:45.792793 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.792139 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:21:45.794953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:45.794926 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-9b0f0d/premium-w2-9b0f0d-kserve-c7bcdcd7f-8mb55"] Nov 04 02:21:46.426443 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:46.426405 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" path="/var/lib/kubelet/pods/64686bc7-a247-4ee9-a28b-3f2fb11aa78a/volumes" Nov 04 02:21:50.265448 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.265373 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" podUID="903a542c-58f6-484a-92f2-c943aca4cc28" containerName="main" containerID="cri-o://bf8e25ff971b4717b0b5127d25ae35cf9cd9757fa22f9b37b1dd36733418aff6" gracePeriod=45 Nov 04 02:21:50.596392 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.595769 6073 generic.go:356] "Generic (PLEG): container finished" podID="903a542c-58f6-484a-92f2-c943aca4cc28" containerID="bf8e25ff971b4717b0b5127d25ae35cf9cd9757fa22f9b37b1dd36733418aff6" exitCode=0 Nov 04 02:21:50.596392 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.595859 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" event={"ID":"903a542c-58f6-484a-92f2-c943aca4cc28","Type":"ContainerDied","Data":"bf8e25ff971b4717b0b5127d25ae35cf9cd9757fa22f9b37b1dd36733418aff6"} Nov 04 02:21:50.620379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.620321 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:21:50.621366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.621320 6073 cpu_manager.go:414] "containerMap: removing container" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerName="main" Nov 04 02:21:50.621524 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.621509 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerName="main" Nov 04 02:21:50.621693 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.621679 6073 cpu_manager.go:414] "containerMap: removing container" podUID="d44836a2-f66f-470f-9d43-1b11e3cd82cf" containerName="revoke-keys" Nov 04 02:21:50.621799 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.621787 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d44836a2-f66f-470f-9d43-1b11e3cd82cf" containerName="revoke-keys" Nov 04 02:21:50.622030 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.622015 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="d44836a2-f66f-470f-9d43-1b11e3cd82cf" containerName="revoke-keys" Nov 04 02:21:50.622143 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.622130 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="64686bc7-a247-4ee9-a28b-3f2fb11aa78a" containerName="main" Nov 04 02:21:50.667416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.667379 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.681288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.680857 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:21:50.735245 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.733701 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e34b54\"/\"distinct-2-w0-e34b54-kserve-self-signed-certs\"" Nov 04 02:21:50.756723 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.756635 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785476 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785575 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\" (UniqueName: \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785722 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785781 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785843 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\" (UniqueName: \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.785895 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\") pod \"903a542c-58f6-484a-92f2-c943aca4cc28\" (UID: \"903a542c-58f6-484a-92f2-c943aca4cc28\") " Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786177 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786217 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786254 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786286 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxfmc\" (UniqueName: \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786367 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.786980 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.786426 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.787961 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.787933 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:50.788192 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.788170 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:50.789797 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.789751 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:50.792389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.790056 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:50.836662 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.825441 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "kube-api-access-g6pnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:50.839468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.839429 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs" pod "903a542c-58f6-484a-92f2-c943aca4cc28" (UID: "903a542c-58f6-484a-92f2-c943aca4cc28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:21:50.887450 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887420 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887450 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887456 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887687 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887478 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887687 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887501 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxfmc\" (UniqueName: \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887687 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887552 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887687 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887613 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.887922 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887898 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.888001 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.887946 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.888050 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888019 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.888154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888136 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.888389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888160 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.888389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888177 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6pnl\" (UniqueName: \"kubernetes.io/projected/903a542c-58f6-484a-92f2-c943aca4cc28-kube-api-access-g6pnl\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.888389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888191 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.888389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888206 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/903a542c-58f6-484a-92f2-c943aca4cc28-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.888389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.888219 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/903a542c-58f6-484a-92f2-c943aca4cc28-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:50.889916 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.889890 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.890235 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.890214 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.959190 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.959103 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxfmc\" (UniqueName: \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\") pod \"distinct-2-w0-e34b54-kserve-cc444dd97-6brmh\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:50.988288 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:50.988245 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:51.467405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.467366 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:21:51.469201 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:21:51.469173 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86ca4814_6244_4efa_af89_752b08566662.slice/crio-59ab603a8159838132c7b9d41c70213cc723618478d7e6b343670548ec3403ae WatchSource:0}: Error finding container 59ab603a8159838132c7b9d41c70213cc723618478d7e6b343670548ec3403ae: Status 404 returned error can't find the container with id 59ab603a8159838132c7b9d41c70213cc723618478d7e6b343670548ec3403ae Nov 04 02:21:51.604019 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.603978 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" event={"ID":"86ca4814-6244-4efa-af89-752b08566662","Type":"ContainerStarted","Data":"f5c33d3a918e16058f7aa5d92e83f08f9cf798fed6fc9911479575648972e445"} Nov 04 02:21:51.604210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.604033 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" event={"ID":"86ca4814-6244-4efa-af89-752b08566662","Type":"ContainerStarted","Data":"59ab603a8159838132c7b9d41c70213cc723618478d7e6b343670548ec3403ae"} Nov 04 02:21:51.609371 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.605770 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" Nov 04 02:21:51.609371 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.605774 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn" event={"ID":"903a542c-58f6-484a-92f2-c943aca4cc28","Type":"ContainerDied","Data":"59f7375c4dc32480785eb43c128497ca36aac0ef61f1a9ab7dc363d464efb956"} Nov 04 02:21:51.609371 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.605823 6073 scope.go:122] "RemoveContainer" containerID="bf8e25ff971b4717b0b5127d25ae35cf9cd9757fa22f9b37b1dd36733418aff6" Nov 04 02:21:51.725805 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.725699 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" podStartSLOduration=1.7256811060000001 podStartE2EDuration="1.725681106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:51.723927621 +0000 UTC m=+889.419617821" watchObservedRunningTime="2026-11-04 02:21:51.725681106 +0000 UTC m=+889.421371306" Nov 04 02:21:51.847708 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.847676 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:21:51.850219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.850191 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-5a27ba/premium-w3-5a27ba-kserve-75c4575867-blhhn"] Nov 04 02:21:51.988580 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:51.988504 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:21:52.432128 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:21:52.432098 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="903a542c-58f6-484a-92f2-c943aca4cc28" path="/var/lib/kubelet/pods/903a542c-58f6-484a-92f2-c943aca4cc28/volumes" Nov 04 02:21:53.858084 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:21:53.858051 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:22:00.989066 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:00.989032 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:22:01.002126 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:01.002101 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:22:01.687296 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:01.687258 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:22:07.723641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:07.723606 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:22:07.724070 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:07.723817 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" containerID="cri-o://2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002" gracePeriod=30 Nov 04 02:22:13.011969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.011943 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:22:13.166536 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166504 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166553 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166588 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166613 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166718 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166779 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166801 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166853 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166878 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\") pod \"0eb41225-5e16-40d9-9206-32f5835e5b0f\" (UID: \"0eb41225-5e16-40d9-9206-32f5835e5b0f\") " Nov 04 02:22:13.166969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.166954 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:13.167218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167056 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:13.167218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167164 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:13.167218 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167202 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:13.167410 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167367 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:13.167576 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167556 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.167649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167583 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.167649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167599 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.167649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167613 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eb41225-5e16-40d9-9206-32f5835e5b0f-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.167649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.167627 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.169160 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.169139 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:13.169297 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.169279 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:13.169508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.169485 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "kube-api-access-2tjvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:13.169717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.169696 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token" pod "0eb41225-5e16-40d9-9206-32f5835e5b0f" (UID: "0eb41225-5e16-40d9-9206-32f5835e5b0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:13.269238 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.269210 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.269238 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.269240 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.269486 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.269256 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tjvz\" (UniqueName: \"kubernetes.io/projected/0eb41225-5e16-40d9-9206-32f5835e5b0f-kube-api-access-2tjvz\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.269486 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.269270 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eb41225-5e16-40d9-9206-32f5835e5b0f-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:13.752178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.752078 6073 generic.go:356] "Generic (PLEG): container finished" podID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerID="2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002" exitCode=0 Nov 04 02:22:13.752178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.752117 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" Nov 04 02:22:13.752178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.752164 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" event={"ID":"0eb41225-5e16-40d9-9206-32f5835e5b0f","Type":"ContainerDied","Data":"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002"} Nov 04 02:22:13.752540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.752216 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp" event={"ID":"0eb41225-5e16-40d9-9206-32f5835e5b0f","Type":"ContainerDied","Data":"3730fc3a73652eb2055b697453e599cbd49a9fb61a26b29788f69e6a507719df"} Nov 04 02:22:13.752540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.752240 6073 scope.go:122] "RemoveContainer" containerID="2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002" Nov 04 02:22:13.766029 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.766008 6073 scope.go:122] "RemoveContainer" containerID="2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002" Nov 04 02:22:13.766369 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:13.766346 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002\": container with ID starting with 2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002 not found: ID does not exist" containerID="2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002" Nov 04 02:22:13.766426 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.766380 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002"} err="failed to get container status \"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002\": rpc error: code = NotFound desc = could not find container \"2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002\": container with ID starting with 2d498bf6b79309b8536b6d28eb62508a7d9f1301315e25ee70984a4d492ef002 not found: ID does not exist" Nov 04 02:22:13.971848 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:13.971815 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:22:14.022266 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:14.022237 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-s2hxp"] Nov 04 02:22:14.433425 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:14.431835 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" path="/var/lib/kubelet/pods/0eb41225-5e16-40d9-9206-32f5835e5b0f/volumes" Nov 04 02:22:15.511044 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:15.510999 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:22:15.511625 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:15.511252 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" podUID="4af1abc7-9563-46e8-9a54-138c336d4563" containerName="maas-api" containerID="cri-o://23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce" gracePeriod=30 Nov 04 02:22:16.786358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.786321 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:22:16.801379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.801314 6073 generic.go:356] "Generic (PLEG): container finished" podID="4af1abc7-9563-46e8-9a54-138c336d4563" containerID="23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce" exitCode=0 Nov 04 02:22:16.801379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.801368 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" Nov 04 02:22:16.801584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.801396 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" event={"ID":"4af1abc7-9563-46e8-9a54-138c336d4563","Type":"ContainerDied","Data":"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce"} Nov 04 02:22:16.801584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.801451 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67" event={"ID":"4af1abc7-9563-46e8-9a54-138c336d4563","Type":"ContainerDied","Data":"66abd7c603ff5c7cb40557e421f36fca8217b51616b35622bc0541443d120972"} Nov 04 02:22:16.801584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.801467 6073 scope.go:122] "RemoveContainer" containerID="23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce" Nov 04 02:22:16.810658 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.810638 6073 scope.go:122] "RemoveContainer" containerID="23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce" Nov 04 02:22:16.810943 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:16.810921 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce\": container with ID starting with 23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce not found: ID does not exist" containerID="23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce" Nov 04 02:22:16.810993 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.810954 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce"} err="failed to get container status \"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce\": rpc error: code = NotFound desc = could not find container \"23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce\": container with ID starting with 23f1ab89656717bb4b040c1d8ba45aa3f9fc7bb4d54f836bd0dec1b537708cce not found: ID does not exist" Nov 04 02:22:16.843812 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.843744 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\") pod \"4af1abc7-9563-46e8-9a54-138c336d4563\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " Nov 04 02:22:16.843952 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.843857 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\") pod \"4af1abc7-9563-46e8-9a54-138c336d4563\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " Nov 04 02:22:16.843952 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.843939 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\" (UniqueName: \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\") pod \"4af1abc7-9563-46e8-9a54-138c336d4563\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " Nov 04 02:22:16.844070 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.844007 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\") pod \"4af1abc7-9563-46e8-9a54-138c336d4563\" (UID: \"4af1abc7-9563-46e8-9a54-138c336d4563\") " Nov 04 02:22:16.844261 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.844239 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca" pod "4af1abc7-9563-46e8-9a54-138c336d4563" (UID: "4af1abc7-9563-46e8-9a54-138c336d4563"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:16.844432 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.844416 6073 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4af1abc7-9563-46e8-9a54-138c336d4563-openshift-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:16.846078 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.846056 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls" pod "4af1abc7-9563-46e8-9a54-138c336d4563" (UID: "4af1abc7-9563-46e8-9a54-138c336d4563"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:16.846173 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.846135 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4" pod "4af1abc7-9563-46e8-9a54-138c336d4563" (UID: "4af1abc7-9563-46e8-9a54-138c336d4563"). InnerVolumeSpecName "kube-api-access-p7mf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:16.846173 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.846147 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs" pod "4af1abc7-9563-46e8-9a54-138c336d4563" (UID: "4af1abc7-9563-46e8-9a54-138c336d4563"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:16.945397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.945363 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7mf4\" (UniqueName: \"kubernetes.io/projected/4af1abc7-9563-46e8-9a54-138c336d4563-kube-api-access-p7mf4\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:16.945397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.945396 6073 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-metrics-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:16.945593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:16.945408 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4af1abc7-9563-46e8-9a54-138c336d4563-maas-api-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:17.276704 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:17.276668 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:22:17.328968 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:17.328935 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf267-c97dc668d-5dc67"] Nov 04 02:22:18.426599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:18.426566 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4af1abc7-9563-46e8-9a54-138c336d4563" path="/var/lib/kubelet/pods/4af1abc7-9563-46e8-9a54-138c336d4563/volumes" Nov 04 02:22:21.520757 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:21.520724 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:22:21.521179 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:21.520904 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" containerID="cri-o://01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3" gracePeriod=30 Nov 04 02:22:23.184904 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.184871 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:22:23.185391 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.185064 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" podUID="9602d0ea-18a5-4973-a680-5834573e5959" containerName="maas-api" containerID="cri-o://f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3" gracePeriod=30 Nov 04 02:22:23.459814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.459708 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:22:23.525321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.525294 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\") pod \"9602d0ea-18a5-4973-a680-5834573e5959\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " Nov 04 02:22:23.525500 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.525359 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\" (UniqueName: \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\") pod \"9602d0ea-18a5-4973-a680-5834573e5959\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " Nov 04 02:22:23.525500 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.525447 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\") pod \"9602d0ea-18a5-4973-a680-5834573e5959\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " Nov 04 02:22:23.525500 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.525481 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\") pod \"9602d0ea-18a5-4973-a680-5834573e5959\" (UID: \"9602d0ea-18a5-4973-a680-5834573e5959\") " Nov 04 02:22:23.525863 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.525832 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca" pod "9602d0ea-18a5-4973-a680-5834573e5959" (UID: "9602d0ea-18a5-4973-a680-5834573e5959"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:23.527623 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.527601 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs" pod "9602d0ea-18a5-4973-a680-5834573e5959" (UID: "9602d0ea-18a5-4973-a680-5834573e5959"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:23.527708 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.527670 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b" pod "9602d0ea-18a5-4973-a680-5834573e5959" (UID: "9602d0ea-18a5-4973-a680-5834573e5959"). InnerVolumeSpecName "kube-api-access-hpm9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:23.527747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.527722 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls" pod "9602d0ea-18a5-4973-a680-5834573e5959" (UID: "9602d0ea-18a5-4973-a680-5834573e5959"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:23.627551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.627519 6073 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9602d0ea-18a5-4973-a680-5834573e5959-openshift-service-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.627551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.627547 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-maas-api-tls\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.627551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.627557 6073 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9602d0ea-18a5-4973-a680-5834573e5959-metrics-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.627778 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.627568 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpm9b\" (UniqueName: \"kubernetes.io/projected/9602d0ea-18a5-4973-a680-5834573e5959-kube-api-access-hpm9b\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.845422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.845378 6073 generic.go:356] "Generic (PLEG): container finished" podID="9602d0ea-18a5-4973-a680-5834573e5959" containerID="f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3" exitCode=0 Nov 04 02:22:23.845599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.845435 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" Nov 04 02:22:23.845599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.845452 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" event={"ID":"9602d0ea-18a5-4973-a680-5834573e5959","Type":"ContainerDied","Data":"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3"} Nov 04 02:22:23.845599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.845511 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5" event={"ID":"9602d0ea-18a5-4973-a680-5834573e5959","Type":"ContainerDied","Data":"51228fb1ed1b10458bddde108c0536d5a401bedd198a644829bb941e4c97e227"} Nov 04 02:22:23.845599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.845535 6073 scope.go:122] "RemoveContainer" containerID="f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3" Nov 04 02:22:23.856791 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.856774 6073 scope.go:122] "RemoveContainer" containerID="f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3" Nov 04 02:22:23.857058 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:23.857031 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3\": container with ID starting with f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3 not found: ID does not exist" containerID="f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3" Nov 04 02:22:23.857116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:23.857060 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3"} err="failed to get container status \"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3\": rpc error: code = NotFound desc = could not find container \"f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3\": container with ID starting with f52384948a25e207ad7eb94f94c0e1e81e965db6ab3a3db2153b29a0b0488ca3 not found: ID does not exist" Nov 04 02:22:23.859968 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:23.859942 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:22:24.022612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:24.022579 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:22:24.025986 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:24.025965 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e970c7-f57648c7b-lvwp5"] Nov 04 02:22:24.426626 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:24.426596 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9602d0ea-18a5-4973-a680-5834573e5959" path="/var/lib/kubelet/pods/9602d0ea-18a5-4973-a680-5834573e5959/volumes" Nov 04 02:22:26.792321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.792262 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:22:26.863849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.863817 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.863904 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.863975 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864021 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864046 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864098 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864275 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864181 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864275 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864263 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864267 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:26.864397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864290 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\") pod \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\" (UID: \"2bd2903b-2f65-4e69-a3e3-09a63082cbc1\") " Nov 04 02:22:26.864397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864387 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:26.864666 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864646 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:26.865089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864789 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.865089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864815 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.865089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.864829 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.865089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.865070 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:26.865577 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.865552 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:26.867199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.867158 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:26.867199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.867186 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:26.868548 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.868521 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:26.869441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.869381 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529" pod "2bd2903b-2f65-4e69-a3e3-09a63082cbc1" (UID: "2bd2903b-2f65-4e69-a3e3-09a63082cbc1"). InnerVolumeSpecName "kube-api-access-gs529". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:26.885807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.885504 6073 generic.go:356] "Generic (PLEG): container finished" podID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerID="01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3" exitCode=0 Nov 04 02:22:26.885807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.885560 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerDied","Data":"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3"} Nov 04 02:22:26.885807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.885589 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" Nov 04 02:22:26.885807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.885618 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf" event={"ID":"2bd2903b-2f65-4e69-a3e3-09a63082cbc1","Type":"ContainerDied","Data":"8df4e5e7c7a92dba78b493ee2d7f021f02c613737d2650ef8dacd0495f829696"} Nov 04 02:22:26.885807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.885648 6073 scope.go:122] "RemoveContainer" containerID="01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3" Nov 04 02:22:26.920074 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.920053 6073 scope.go:122] "RemoveContainer" containerID="99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d" Nov 04 02:22:26.929656 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.929633 6073 scope.go:122] "RemoveContainer" containerID="01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3" Nov 04 02:22:26.929941 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:26.929918 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3\": container with ID starting with 01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3 not found: ID does not exist" containerID="01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3" Nov 04 02:22:26.930053 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.929949 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3"} err="failed to get container status \"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3\": rpc error: code = NotFound desc = could not find container \"01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3\": container with ID starting with 01c6c0c66c1e21a56ea6ceb22a9be3f45bea17fd06633a7cd1513c60a7838ee3 not found: ID does not exist" Nov 04 02:22:26.930053 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.929968 6073 scope.go:122] "RemoveContainer" containerID="99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d" Nov 04 02:22:26.930198 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:26.930178 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d\": container with ID starting with 99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d not found: ID does not exist" containerID="99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d" Nov 04 02:22:26.930257 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.930211 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d"} err="failed to get container status \"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d\": rpc error: code = NotFound desc = could not find container \"99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d\": container with ID starting with 99e6db6ac9527860770f76965f707e023491d3086ae93d9bf05470676b92c27d not found: ID does not exist" Nov 04 02:22:26.966182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966152 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.966182 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966182 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.966381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966194 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.966381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966203 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.966381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966212 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs529\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-kube-api-access-gs529\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:26.966381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:26.966221 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bd2903b-2f65-4e69-a3e3-09a63082cbc1-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:27.069752 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:27.069674 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:22:27.073490 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:27.073461 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9b0f0d-openshift-default-76684f6b87-f7dsf"] Nov 04 02:22:27.953633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:27.953604 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:22:27.954060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:27.953796 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" containerID="cri-o://35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb" gracePeriod=30 Nov 04 02:22:28.425789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:28.425756 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" path="/var/lib/kubelet/pods/2bd2903b-2f65-4e69-a3e3-09a63082cbc1/volumes" Nov 04 02:22:30.139671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:30.139640 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:22:30.140109 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:30.139854 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" containerID="cri-o://b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b" gracePeriod=30 Nov 04 02:22:31.177103 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:31.177073 6073 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 02:22:31.177526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:31.177127 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 02:22:33.209869 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.209844 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:22:33.347136 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347080 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347136 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347140 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347202 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347250 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347296 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347320 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347384 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347419 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347492 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347526 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347557 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\") pod \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\" (UID: \"ddfaacc0-62da-4634-a8c6-4e38f86ffe09\") " Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347582 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:33.347661 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347650 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:33.348011 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.347986 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:33.348207 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.348156 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.348207 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.348180 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.348207 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.348195 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.348366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.348208 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.348366 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.348221 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.349613 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.349586 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "kube-api-access-xxrk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:33.349613 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.349593 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:33.350076 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.350057 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:33.350135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.350071 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token" pod "ddfaacc0-62da-4634-a8c6-4e38f86ffe09" (UID: "ddfaacc0-62da-4634-a8c6-4e38f86ffe09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:33.449647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.449608 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrk6\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-kube-api-access-xxrk6\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.449647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.449640 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.449647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.449653 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.449870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.449665 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddfaacc0-62da-4634-a8c6-4e38f86ffe09-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:33.718432 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.718347 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:22:33.718626 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.718595 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" containerID="cri-o://7f4ff8ca836024b2da144766a561869e4a4cb59c5fd17747cf4ee6f33214cce1" gracePeriod=30 Nov 04 02:22:33.944038 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.944002 6073 generic.go:356] "Generic (PLEG): container finished" podID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerID="35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb" exitCode=0 Nov 04 02:22:33.944246 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.944069 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" Nov 04 02:22:33.944246 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.944094 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerDied","Data":"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb"} Nov 04 02:22:33.944246 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.944147 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f" event={"ID":"ddfaacc0-62da-4634-a8c6-4e38f86ffe09","Type":"ContainerDied","Data":"465bd45e82100dbae0f3a37961f939c1081d99143136ea7277d248ba0e4029bb"} Nov 04 02:22:33.944246 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.944168 6073 scope.go:122] "RemoveContainer" containerID="35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb" Nov 04 02:22:33.954455 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.954431 6073 scope.go:122] "RemoveContainer" containerID="1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e" Nov 04 02:22:33.963153 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.963119 6073 scope.go:122] "RemoveContainer" containerID="35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb" Nov 04 02:22:33.963440 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:33.963418 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb\": container with ID starting with 35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb not found: ID does not exist" containerID="35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb" Nov 04 02:22:33.963496 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.963451 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb"} err="failed to get container status \"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb\": rpc error: code = NotFound desc = could not find container \"35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb\": container with ID starting with 35f60a8ca03291c234ebfc524955386fbdb9b79d0ff6218ad170eeea3cad09fb not found: ID does not exist" Nov 04 02:22:33.963496 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.963468 6073 scope.go:122] "RemoveContainer" containerID="1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e" Nov 04 02:22:33.963719 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:33.963698 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e\": container with ID starting with 1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e not found: ID does not exist" containerID="1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e" Nov 04 02:22:33.963759 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:33.963726 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e"} err="failed to get container status \"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e\": rpc error: code = NotFound desc = could not find container \"1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e\": container with ID starting with 1f3f7ffab0cc0a994ff1fe06067c5e90766a1bc00914c739a9521f2a0bfb620e not found: ID does not exist" Nov 04 02:22:34.108398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.108365 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:22:34.112021 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.111998 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-czs4f"] Nov 04 02:22:34.426464 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.426373 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" path="/var/lib/kubelet/pods/ddfaacc0-62da-4634-a8c6-4e38f86ffe09/volumes" Nov 04 02:22:34.493011 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.492979 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:22:34.493619 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493596 6073 cpu_manager.go:414] "containerMap: removing container" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.493619 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493618 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493636 6073 cpu_manager.go:414] "containerMap: removing container" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493643 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493661 6073 cpu_manager.go:414] "containerMap: removing container" podUID="903a542c-58f6-484a-92f2-c943aca4cc28" containerName="main" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493669 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="903a542c-58f6-484a-92f2-c943aca4cc28" containerName="main" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493684 6073 cpu_manager.go:414] "containerMap: removing container" podUID="4af1abc7-9563-46e8-9a54-138c336d4563" containerName="maas-api" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493692 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4af1abc7-9563-46e8-9a54-138c336d4563" containerName="maas-api" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493714 6073 cpu_manager.go:414] "containerMap: removing container" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493722 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493731 6073 cpu_manager.go:414] "containerMap: removing container" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493737 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493750 6073 cpu_manager.go:414] "containerMap: removing container" podUID="9602d0ea-18a5-4973-a680-5834573e5959" containerName="maas-api" Nov 04 02:22:34.493792 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493758 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9602d0ea-18a5-4973-a680-5834573e5959" containerName="maas-api" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493889 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493905 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="0eb41225-5e16-40d9-9206-32f5835e5b0f" containerName="istio-proxy" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493918 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="903a542c-58f6-484a-92f2-c943aca4cc28" containerName="main" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493930 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="4af1abc7-9563-46e8-9a54-138c336d4563" containerName="maas-api" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493939 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="2bd2903b-2f65-4e69-a3e3-09a63082cbc1" containerName="istio-proxy" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493948 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493957 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="9602d0ea-18a5-4973-a680-5834573e5959" containerName="maas-api" Nov 04 02:22:34.494357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.493966 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:34.498803 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.498785 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.552083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.552006 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"openshift-service-ca.crt\"" Nov 04 02:22:34.552083 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.552053 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"kube-root-ca.crt\"" Nov 04 02:22:34.552436 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.552168 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"distinct-2-w6-9adedd-kserve-self-signed-certs\"" Nov 04 02:22:34.552436 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.552369 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"default-dockercfg-jk7zs\"" Nov 04 02:22:34.554757 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.554732 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:22:34.662809 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.662772 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.662960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.662815 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpzjv\" (UniqueName: \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.663028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.662952 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.663028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.662983 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.663028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.663016 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.663161 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.663145 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764272 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764239 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764507 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764279 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764507 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764324 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764507 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764454 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764675 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764518 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764675 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764641 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpzjv\" (UniqueName: \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764746 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764846 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764795 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.764899 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.764848 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.766785 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.766765 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.767170 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.767149 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:34.825246 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:34.825207 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpzjv\" (UniqueName: \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\") pod \"distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:35.110389 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.110323 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:35.380061 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.380038 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:22:35.474518 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474488 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474528 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474555 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474639 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474712 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474771 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474808 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474848 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474888 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:35.475012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.474918 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\") pod \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\" (UID: \"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94\") " Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475048 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475103 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475204 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475630 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475652 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475667 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.475728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475682 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.476105 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.475766 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:35.476970 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.476950 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:35.477084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.476980 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:35.477084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.477031 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:35.477163 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.477130 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz" pod "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" (UID: "1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94"). InnerVolumeSpecName "kube-api-access-qd5zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:35.485951 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.485930 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:22:35.487359 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:22:35.487321 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6fcbfc6_eae2_4763_8c98_648130f8ff6f.slice/crio-52a6b23003dc8cd8d8035fe33d0712f9a9325c791378bdc9eafb7b9416784049 WatchSource:0}: Error finding container 52a6b23003dc8cd8d8035fe33d0712f9a9325c791378bdc9eafb7b9416784049: Status 404 returned error can't find the container with id 52a6b23003dc8cd8d8035fe33d0712f9a9325c791378bdc9eafb7b9416784049 Nov 04 02:22:35.576970 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.576928 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.576970 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.576968 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.577225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.576986 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.577225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.576999 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.577225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.577014 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd5zz\" (UniqueName: \"kubernetes.io/projected/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94-kube-api-access-qd5zz\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:35.957420 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.957377 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" event={"ID":"e6fcbfc6-eae2-4763-8c98-648130f8ff6f","Type":"ContainerStarted","Data":"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813"} Nov 04 02:22:35.957420 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.957425 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" event={"ID":"e6fcbfc6-eae2-4763-8c98-648130f8ff6f","Type":"ContainerStarted","Data":"52a6b23003dc8cd8d8035fe33d0712f9a9325c791378bdc9eafb7b9416784049"} Nov 04 02:22:35.959057 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.959032 6073 generic.go:356] "Generic (PLEG): container finished" podID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerID="b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b" exitCode=0 Nov 04 02:22:35.959190 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.959143 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" Nov 04 02:22:35.959263 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.959180 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerDied","Data":"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b"} Nov 04 02:22:35.959318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.959276 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm" event={"ID":"1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94","Type":"ContainerDied","Data":"18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0"} Nov 04 02:22:35.959318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.959310 6073 scope.go:122] "RemoveContainer" containerID="b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b" Nov 04 02:22:35.972124 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.972105 6073 scope.go:122] "RemoveContainer" containerID="cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea" Nov 04 02:22:35.981958 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.981936 6073 scope.go:122] "RemoveContainer" containerID="b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b" Nov 04 02:22:35.982264 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:35.982246 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b\": container with ID starting with b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b not found: ID does not exist" containerID="b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b" Nov 04 02:22:35.982348 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.982276 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b"} err="failed to get container status \"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b\": rpc error: code = NotFound desc = could not find container \"b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b\": container with ID starting with b3737daf0b1017ff78a4120eb6a5c88c090a27559115b15a413d33b59d449d8b not found: ID does not exist" Nov 04 02:22:35.982348 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.982301 6073 scope.go:122] "RemoveContainer" containerID="cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea" Nov 04 02:22:35.982592 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:35.982566 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea\": container with ID starting with cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea not found: ID does not exist" containerID="cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea" Nov 04 02:22:35.982639 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:35.982603 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea"} err="failed to get container status \"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea\": rpc error: code = NotFound desc = could not find container \"cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea\": container with ID starting with cbcaa43c42d4b7806de853e1024b7400cfaaf142914821f86afb4b20230d8bea not found: ID does not exist" Nov 04 02:22:36.004517 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:36.004474 6073 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1d2cf9_6cb5_47a0_b3f8_34f5d0ceaf94.slice/crio-18376c0c3c7206feb729fc324747f55bb352cc8c9af06d1dd5b3aa207226afb0\": RecentStats: unable to find data in memory cache]" Nov 04 02:22:36.076595 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:36.076497 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" podStartSLOduration=2.076477045 podStartE2EDuration="2.076477045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:36.074765352 +0000 UTC m=+933.770455564" watchObservedRunningTime="2026-11-04 02:22:36.076477045 +0000 UTC m=+933.772167244" Nov 04 02:22:36.110861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:36.110830 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:36.188313 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:36.187780 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:22:36.192735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:36.192636 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-9knfm"] Nov 04 02:22:36.426647 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:36.426572 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" path="/var/lib/kubelet/pods/1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94/volumes" Nov 04 02:22:38.980283 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.980257 6073 generic.go:356] "Generic (PLEG): container finished" podID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerID="7f4ff8ca836024b2da144766a561869e4a4cb59c5fd17747cf4ee6f33214cce1" exitCode=0 Nov 04 02:22:38.980780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.980362 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerDied","Data":"7f4ff8ca836024b2da144766a561869e4a4cb59c5fd17747cf4ee6f33214cce1"} Nov 04 02:22:38.980780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.980434 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" event={"ID":"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf","Type":"ContainerDied","Data":"d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5"} Nov 04 02:22:38.980780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.980461 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d317ba7b21e180ddd23b7486a378eee75744fb7899776373f6e4aae6262381f5" Nov 04 02:22:38.980780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.980479 6073 scope.go:122] "RemoveContainer" containerID="491b286cfa5933f86b8518dd0910bf4aca9acf4f9b63f7e8ac4f0cc61fbea8c7" Nov 04 02:22:38.988297 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:38.988241 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:22:39.014767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.014735 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.014928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.014782 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.014928 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.014819 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015050 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.014933 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015050 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.014995 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015092 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015096 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:39.015249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015146 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015212 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.015379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015231 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:39.015379 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015304 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:39.015499 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015389 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:39.015657 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.015634 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\") pod \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\" (UID: \"c4e50be5-0c8e-45ec-8b8f-ba696163f1cf\") " Nov 04 02:22:39.016019 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.016005 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.016111 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.016014 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:39.016111 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.016024 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.016111 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.016039 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.016111 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.016049 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.017318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.017293 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "kube-api-access-rx8jz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:39.017495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.017477 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:39.017591 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.017572 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:39.017641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.017624 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token" pod "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" (UID: "c4e50be5-0c8e-45ec-8b8f-ba696163f1cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:39.117505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.117471 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.117505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.117499 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.117505 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.117509 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx8jz\" (UniqueName: \"kubernetes.io/projected/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-kube-api-access-rx8jz\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.117735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.117520 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.117735 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.117529 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:39.988361 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:39.988301 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl" Nov 04 02:22:40.273871 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.273832 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:22:40.277682 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.277657 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e970c7-openshift-default-f69bbbfc9-5z6tl"] Nov 04 02:22:40.426237 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.426202 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" path="/var/lib/kubelet/pods/c4e50be5-0c8e-45ec-8b8f-ba696163f1cf/volumes" Nov 04 02:22:40.764200 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764125 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:22:40.764578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764565 6073 cpu_manager.go:414] "containerMap: removing container" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:40.764634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764580 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:40.764634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764598 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:40.764634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764604 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:40.764634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764610 6073 cpu_manager.go:414] "containerMap: removing container" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:40.764634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764615 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddfaacc0-62da-4634-a8c6-4e38f86ffe09" containerName="istio-proxy" Nov 04 02:22:40.764799 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764697 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:40.764799 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.764705 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:40.769034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.769019 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.830888 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.830859 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:22:40.833674 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833640 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.833794 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833700 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.833794 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833790 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.833878 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833807 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cls4g\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.833878 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833862 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.834007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.833986 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.834057 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.834038 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.834198 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.834178 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.834263 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.834216 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935177 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935148 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935385 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935192 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935385 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935229 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935385 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935246 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cls4g\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935385 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935301 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935617 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935474 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935617 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935530 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935617 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935605 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935764 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935611 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.935764 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935636 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.936023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935974 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.936023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.935997 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.936412 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.936177 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.936412 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.936307 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.937889 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.937866 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.937975 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.937949 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.996085 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.996052 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cls4g\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:40.996545 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:40.996458 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:41.082672 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:41.082640 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:41.511601 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:41.511574 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:22:41.513277 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:22:41.513251 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaea0781_fac2_4f39_86c1_a250ccb6df4b.slice/crio-2c1ff49d94c060ee9df28d62fbe0eed9701074632912afb3821c2683e42c7ccd WatchSource:0}: Error finding container 2c1ff49d94c060ee9df28d62fbe0eed9701074632912afb3821c2683e42c7ccd: Status 404 returned error can't find the container with id 2c1ff49d94c060ee9df28d62fbe0eed9701074632912afb3821c2683e42c7ccd Nov 04 02:22:41.515490 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:41.515458 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:22:41.515588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:41.515524 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:22:41.515588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:41.515551 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:22:42.019215 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.019167 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" event={"ID":"caea0781-fac2-4f39-86c1-a250ccb6df4b","Type":"ContainerStarted","Data":"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b"} Nov 04 02:22:42.019671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.019237 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" event={"ID":"caea0781-fac2-4f39-86c1-a250ccb6df4b","Type":"ContainerStarted","Data":"2c1ff49d94c060ee9df28d62fbe0eed9701074632912afb3821c2683e42c7ccd"} Nov 04 02:22:42.083682 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.083637 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:42.085194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.085163 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:42.085298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.085205 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:42.141427 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:42.141380 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podStartSLOduration=2.141364898 podStartE2EDuration="2.141364898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:42.138528683 +0000 UTC m=+939.834218885" watchObservedRunningTime="2026-11-04 02:22:42.141364898 +0000 UTC m=+939.837055095" Nov 04 02:22:43.083911 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:43.083878 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:43.084321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:43.083939 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:44.083713 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:44.083670 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:44.083865 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:44.083741 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:45.083887 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:45.083850 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:45.084393 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:45.083910 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:45.116358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:45.113710 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:45.128101 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:45.128077 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:46.063895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:46.063867 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:22:46.083395 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:46.083366 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:46.083523 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:46.083411 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:47.083321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:47.083282 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:47.083861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:47.083350 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:48.083539 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:48.083503 6073 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Nov 04 02:22:48.084181 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:48.083568 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Nov 04 02:22:49.087427 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:49.087401 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:49.087804 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:49.087740 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:49.088475 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:49.088458 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:22:50.514057 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514024 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd"] Nov 04 02:22:50.514481 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514468 6073 cpu_manager.go:414] "containerMap: removing container" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:50.514525 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514482 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:50.514525 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514500 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:50.514525 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514505 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:50.514623 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514589 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4e50be5-0c8e-45ec-8b8f-ba696163f1cf" containerName="istio-proxy" Nov 04 02:22:50.514623 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.514599 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e1d2cf9-6cb5-47a0-b3f8-34f5d0ceaf94" containerName="istio-proxy" Nov 04 02:22:50.517436 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.517419 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.572851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.572573 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:22:50.621849 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.621817 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:22:50.622015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.621874 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:22:50.622057 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.622011 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:22:50.624251 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.624231 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd"] Nov 04 02:22:50.679220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.679195 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj8qs\" (UniqueName: \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.679419 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.679248 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.679419 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.679395 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.780169 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.780133 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj8qs\" (UniqueName: \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.780396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.780224 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.780396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.780321 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.781005 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.780981 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.782714 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.782690 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:50.841621 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:50.841595 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj8qs\" (UniqueName: \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\") pod \"maas-api-revoke-sub-b545d79e-mtxjd\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:51.131807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:51.131736 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:51.505449 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:51.505416 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd"] Nov 04 02:22:51.507064 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:22:51.507032 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a57895_a14c_43dc_a70f_204eec6a274f.slice/crio-2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242 WatchSource:0}: Error finding container 2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242: Status 404 returned error can't find the container with id 2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242 Nov 04 02:22:52.096447 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:52.096358 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" event={"ID":"37a57895-a14c-43dc-a70f-204eec6a274f","Type":"ContainerStarted","Data":"c2ec1466e986f786aa1e60c1c7f8a0743504b301e32fbdb7b16c685eead64819"} Nov 04 02:22:52.096447 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:52.096410 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" event={"ID":"37a57895-a14c-43dc-a70f-204eec6a274f","Type":"ContainerStarted","Data":"2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242"} Nov 04 02:22:52.211606 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:52.211554 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" podStartSLOduration=2.211535231 podStartE2EDuration="2.211535231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:52.211309101 +0000 UTC m=+949.906999302" watchObservedRunningTime="2026-11-04 02:22:52.211535231 +0000 UTC m=+949.907225433" Nov 04 02:22:53.103566 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:53.103528 6073 generic.go:356] "Generic (PLEG): container finished" podID="37a57895-a14c-43dc-a70f-204eec6a274f" containerID="c2ec1466e986f786aa1e60c1c7f8a0743504b301e32fbdb7b16c685eead64819" exitCode=0 Nov 04 02:22:53.104004 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:53.103614 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" event={"ID":"37a57895-a14c-43dc-a70f-204eec6a274f","Type":"ContainerDied","Data":"c2ec1466e986f786aa1e60c1c7f8a0743504b301e32fbdb7b16c685eead64819"} Nov 04 02:22:53.857924 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:22:53.857886 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:22:54.252612 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.252587 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:22:54.427016 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.426939 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\") pod \"37a57895-a14c-43dc-a70f-204eec6a274f\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " Nov 04 02:22:54.427179 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.427034 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\" (UniqueName: \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\") pod \"37a57895-a14c-43dc-a70f-204eec6a274f\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " Nov 04 02:22:54.427179 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.427071 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\") pod \"37a57895-a14c-43dc-a70f-204eec6a274f\" (UID: \"37a57895-a14c-43dc-a70f-204eec6a274f\") " Nov 04 02:22:54.427388 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.427361 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca" pod "37a57895-a14c-43dc-a70f-204eec6a274f" (UID: "37a57895-a14c-43dc-a70f-204eec6a274f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:54.427533 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.427504 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:54.429692 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.429668 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca" pod "37a57895-a14c-43dc-a70f-204eec6a274f" (UID: "37a57895-a14c-43dc-a70f-204eec6a274f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:54.429796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.429773 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs" pod "37a57895-a14c-43dc-a70f-204eec6a274f" (UID: "37a57895-a14c-43dc-a70f-204eec6a274f"). InnerVolumeSpecName "kube-api-access-zj8qs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:54.528278 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.528244 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj8qs\" (UniqueName: \"kubernetes.io/projected/37a57895-a14c-43dc-a70f-204eec6a274f-kube-api-access-zj8qs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:54.528278 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.528275 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37a57895-a14c-43dc-a70f-204eec6a274f-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:54.851996 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.851960 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:22:54.865669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:54.865639 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:22:55.117956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:55.117874 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" event={"ID":"37a57895-a14c-43dc-a70f-204eec6a274f","Type":"ContainerDied","Data":"2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242"} Nov 04 02:22:55.117956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:55.117920 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c7ad0d50378c40ee05efc95350865f606283f59c830bf6859fd26c032086242" Nov 04 02:22:55.117956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:22:55.117922 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b545d79e-mtxjd" Nov 04 02:23:07.064012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.063977 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m"] Nov 04 02:23:07.064861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.064451 6073 cpu_manager.go:414] "containerMap: removing container" podUID="37a57895-a14c-43dc-a70f-204eec6a274f" containerName="revoke-keys" Nov 04 02:23:07.064861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.064464 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37a57895-a14c-43dc-a70f-204eec6a274f" containerName="revoke-keys" Nov 04 02:23:07.064861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.064576 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="37a57895-a14c-43dc-a70f-204eec6a274f" containerName="revoke-keys" Nov 04 02:23:07.069394 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.069377 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.124930 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.124700 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:23:07.158219 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.158189 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.158390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.158236 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.158448 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.158415 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrzxm\" (UniqueName: \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.173953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.173927 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:23:07.174091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.174011 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:23:07.174091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.174070 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:23:07.176764 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.176744 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m"] Nov 04 02:23:07.259049 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.259016 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.259228 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.259059 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.259228 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.259102 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrzxm\" (UniqueName: \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.259740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.259722 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.261554 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.261533 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.319783 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.319708 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrzxm\" (UniqueName: \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\") pod \"maas-api-revoke-sub-d5ae9488-6fg4m\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.382717 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.382685 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:07.761042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:07.761019 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m"] Nov 04 02:23:07.763320 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:07.763290 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebeca20a_2256_4382_abc3_2ca428041bb0.slice/crio-6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744 WatchSource:0}: Error finding container 6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744: Status 404 returned error can't find the container with id 6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744 Nov 04 02:23:08.200102 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.200067 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" event={"ID":"ebeca20a-2256-4382-abc3-2ca428041bb0","Type":"ContainerStarted","Data":"b2b2ffee981623041d84bf92108ab4f51680c4d6f884a83b10db11b691bca8b9"} Nov 04 02:23:08.200668 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.200122 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" event={"ID":"ebeca20a-2256-4382-abc3-2ca428041bb0","Type":"ContainerStarted","Data":"6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744"} Nov 04 02:23:08.213664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.213636 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:23:08.217424 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.217406 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.271922 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.271889 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272101 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.271955 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272184 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272164 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272233 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272195 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272291 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272281 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272493 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272460 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272537 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272603 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272570 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.272603 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.272591 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdq59\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.307743 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.307717 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:23:08.316531 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.316466 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" podStartSLOduration=1.3164476600000001 podStartE2EDuration="1.31644766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:08.314278281 +0000 UTC m=+966.009968482" watchObservedRunningTime="2026-11-04 02:23:08.31644766 +0000 UTC m=+966.012137861" Nov 04 02:23:08.373648 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373620 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.373836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373687 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.373836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373733 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.373836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373762 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.373836 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373791 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdq59\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373849 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373876 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373910 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.373939 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.374084 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.374290 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374424 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.374378 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374540 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.374489 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.374634 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.374613 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.376110 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.376092 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.376431 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.376414 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.460113 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.460069 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdq59\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.501094 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.501060 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.531422 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.531356 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:08.969063 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.969033 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:23:08.969876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.969847 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:23:08.969959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.969903 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:23:08.969959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:08.969931 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:23:09.210727 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.210689 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" event={"ID":"b37ebe37-80b0-408c-b2e8-451e02f0be6e","Type":"ContainerStarted","Data":"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8"} Nov 04 02:23:09.211200 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.210755 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" event={"ID":"b37ebe37-80b0-408c-b2e8-451e02f0be6e","Type":"ContainerStarted","Data":"fdf2d44b88a2aa26366b56aef850a155f78419eb9c00618d2262c31f87111d32"} Nov 04 02:23:09.212690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.212617 6073 generic.go:356] "Generic (PLEG): container finished" podID="ebeca20a-2256-4382-abc3-2ca428041bb0" containerID="b2b2ffee981623041d84bf92108ab4f51680c4d6f884a83b10db11b691bca8b9" exitCode=0 Nov 04 02:23:09.212690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.212664 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" event={"ID":"ebeca20a-2256-4382-abc3-2ca428041bb0","Type":"ContainerDied","Data":"b2b2ffee981623041d84bf92108ab4f51680c4d6f884a83b10db11b691bca8b9"} Nov 04 02:23:09.335003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.334942 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podStartSLOduration=1.3349221390000001 podStartE2EDuration="1.334922139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:09.331595672 +0000 UTC m=+967.027285876" watchObservedRunningTime="2026-11-04 02:23:09.334922139 +0000 UTC m=+967.030612342" Nov 04 02:23:09.532356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.531465 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:09.873118 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.873033 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" podUID="86ca4814-6244-4efa-af89-752b08566662" containerName="main" containerID="cri-o://f5c33d3a918e16058f7aa5d92e83f08f9cf798fed6fc9911479575648972e445" gracePeriod=45 Nov 04 02:23:09.885493 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:09.885458 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" podUID="0db11310-75d3-4c21-a248-691b25092042" containerName="main" containerID="cri-o://b214212f153ca5e97a42a688af0f3b61c081aff06ebee4a126b632c09a8d988f" gracePeriod=45 Nov 04 02:23:10.232671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.231348 6073 generic.go:356] "Generic (PLEG): container finished" podID="0db11310-75d3-4c21-a248-691b25092042" containerID="b214212f153ca5e97a42a688af0f3b61c081aff06ebee4a126b632c09a8d988f" exitCode=0 Nov 04 02:23:10.232671 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.231495 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" event={"ID":"0db11310-75d3-4c21-a248-691b25092042","Type":"ContainerDied","Data":"b214212f153ca5e97a42a688af0f3b61c081aff06ebee4a126b632c09a8d988f"} Nov 04 02:23:10.236359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.233777 6073 generic.go:356] "Generic (PLEG): container finished" podID="86ca4814-6244-4efa-af89-752b08566662" containerID="f5c33d3a918e16058f7aa5d92e83f08f9cf798fed6fc9911479575648972e445" exitCode=0 Nov 04 02:23:10.236359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.234789 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" event={"ID":"86ca4814-6244-4efa-af89-752b08566662","Type":"ContainerDied","Data":"f5c33d3a918e16058f7aa5d92e83f08f9cf798fed6fc9911479575648972e445"} Nov 04 02:23:10.289573 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.289230 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:23:10.352407 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.352288 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.448783 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.448902 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\" (UniqueName: \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.448950 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.448982 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\" (UniqueName: \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449037 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449082 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\" (UniqueName: \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449116 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449152 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449191 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449260 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\") pod \"0db11310-75d3-4c21-a248-691b25092042\" (UID: \"0db11310-75d3-4c21-a248-691b25092042\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449303 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\" (UniqueName: \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.449404 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\") pod \"86ca4814-6244-4efa-af89-752b08566662\" (UID: \"86ca4814-6244-4efa-af89-752b08566662\") " Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.451285 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.452035 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.452314 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.452520 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.454420 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.454441 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.456921 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.454644 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.467981 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.467947 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "kube-api-access-zxfmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:10.470515 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.470486 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:10.471937 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.471908 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "kube-api-access-xzzqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:10.472089 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.472028 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs" pod "86ca4814-6244-4efa-af89-752b08566662" (UID: "86ca4814-6244-4efa-af89-752b08566662"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:10.472398 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.472373 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm" pod "0db11310-75d3-4c21-a248-691b25092042" (UID: "0db11310-75d3-4c21-a248-691b25092042"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:10.536844 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.536566 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:10.536844 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.536623 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550766 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550799 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0db11310-75d3-4c21-a248-691b25092042-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550813 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550827 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ca4814-6244-4efa-af89-752b08566662-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550841 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550857 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzzqj\" (UniqueName: \"kubernetes.io/projected/0db11310-75d3-4c21-a248-691b25092042-kube-api-access-xzzqj\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550871 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550886 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550899 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550912 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0db11310-75d3-4c21-a248-691b25092042-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550926 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxfmc\" (UniqueName: \"kubernetes.io/projected/86ca4814-6244-4efa-af89-752b08566662-kube-api-access-zxfmc\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.551396 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.550941 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ca4814-6244-4efa-af89-752b08566662-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.561208 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.555665 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:10.651709 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.651671 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\" (UniqueName: \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\") pod \"ebeca20a-2256-4382-abc3-2ca428041bb0\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " Nov 04 02:23:10.651878 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.651750 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\") pod \"ebeca20a-2256-4382-abc3-2ca428041bb0\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " Nov 04 02:23:10.651972 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.651953 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\") pod \"ebeca20a-2256-4382-abc3-2ca428041bb0\" (UID: \"ebeca20a-2256-4382-abc3-2ca428041bb0\") " Nov 04 02:23:10.652416 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.652388 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca" pod "ebeca20a-2256-4382-abc3-2ca428041bb0" (UID: "ebeca20a-2256-4382-abc3-2ca428041bb0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:10.652575 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.652546 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.654193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.654162 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca" pod "ebeca20a-2256-4382-abc3-2ca428041bb0" (UID: "ebeca20a-2256-4382-abc3-2ca428041bb0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:10.654562 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.654537 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm" pod "ebeca20a-2256-4382-abc3-2ca428041bb0" (UID: "ebeca20a-2256-4382-abc3-2ca428041bb0"). InnerVolumeSpecName "kube-api-access-vrzxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:10.754194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.754112 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrzxm\" (UniqueName: \"kubernetes.io/projected/ebeca20a-2256-4382-abc3-2ca428041bb0-kube-api-access-vrzxm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:10.754194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:10.754151 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebeca20a-2256-4382-abc3-2ca428041bb0-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:11.239217 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.239180 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" podUID="0db11310-75d3-4c21-a248-691b25092042" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 02:23:11.240886 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.240863 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" Nov 04 02:23:11.240960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.240897 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh" event={"ID":"86ca4814-6244-4efa-af89-752b08566662","Type":"ContainerDied","Data":"59ab603a8159838132c7b9d41c70213cc723618478d7e6b343670548ec3403ae"} Nov 04 02:23:11.240960 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.240943 6073 scope.go:122] "RemoveContainer" containerID="f5c33d3a918e16058f7aa5d92e83f08f9cf798fed6fc9911479575648972e445" Nov 04 02:23:11.242434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.242412 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" Nov 04 02:23:11.242577 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.242456 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5ae9488-6fg4m" event={"ID":"ebeca20a-2256-4382-abc3-2ca428041bb0","Type":"ContainerDied","Data":"6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744"} Nov 04 02:23:11.242577 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.242501 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b14bc8f9c6301a6ff9b8aa91e665e6929665fbcc656d7d228fea8a063261744" Nov 04 02:23:11.244043 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.244023 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" event={"ID":"0db11310-75d3-4c21-a248-691b25092042","Type":"ContainerDied","Data":"22b7a6c5e0a939aa5deed8ccd379883b94c3aa08d7b91754f3744c28646436f5"} Nov 04 02:23:11.244155 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.244119 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h" Nov 04 02:23:11.251405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.251392 6073 scope.go:122] "RemoveContainer" containerID="b214212f153ca5e97a42a688af0f3b61c081aff06ebee4a126b632c09a8d988f" Nov 04 02:23:11.476453 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.476420 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:23:11.480871 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.480845 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/distinct-2-w0-e34b54-kserve-cc444dd97-6brmh"] Nov 04 02:23:11.532551 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.532525 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:11.532728 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.532579 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:11.588385 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.588356 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:23:11.617595 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:11.617542 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e34b54/premium-w0-e34b54-kserve-5f949456d7-2h66h"] Nov 04 02:23:12.425434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:12.425405 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0db11310-75d3-4c21-a248-691b25092042" path="/var/lib/kubelet/pods/0db11310-75d3-4c21-a248-691b25092042/volumes" Nov 04 02:23:12.425823 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:12.425794 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86ca4814-6244-4efa-af89-752b08566662" path="/var/lib/kubelet/pods/86ca4814-6244-4efa-af89-752b08566662/volumes" Nov 04 02:23:12.531778 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:12.531741 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:12.531961 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:12.531806 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:13.532409 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:13.532365 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:13.532829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:13.532421 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:14.532658 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:14.532621 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:14.533149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:14.532683 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:15.533358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:15.532366 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Nov 04 02:23:15.533358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:15.532445 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Nov 04 02:23:16.625941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:16.625907 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:16.626415 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:16.626298 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:16.626979 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:16.626957 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:23:21.130635 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.130600 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b"] Nov 04 02:23:21.131072 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131058 6073 cpu_manager.go:414] "containerMap: removing container" podUID="86ca4814-6244-4efa-af89-752b08566662" containerName="main" Nov 04 02:23:21.131116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131074 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86ca4814-6244-4efa-af89-752b08566662" containerName="main" Nov 04 02:23:21.131116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131091 6073 cpu_manager.go:414] "containerMap: removing container" podUID="0db11310-75d3-4c21-a248-691b25092042" containerName="main" Nov 04 02:23:21.131116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131097 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0db11310-75d3-4c21-a248-691b25092042" containerName="main" Nov 04 02:23:21.131116 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131113 6073 cpu_manager.go:414] "containerMap: removing container" podUID="ebeca20a-2256-4382-abc3-2ca428041bb0" containerName="revoke-keys" Nov 04 02:23:21.131236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131118 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebeca20a-2256-4382-abc3-2ca428041bb0" containerName="revoke-keys" Nov 04 02:23:21.131236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131196 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="0db11310-75d3-4c21-a248-691b25092042" containerName="main" Nov 04 02:23:21.131236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131205 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebeca20a-2256-4382-abc3-2ca428041bb0" containerName="revoke-keys" Nov 04 02:23:21.131236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.131215 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="86ca4814-6244-4efa-af89-752b08566662" containerName="main" Nov 04 02:23:21.134381 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.134367 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.199649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.199622 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.199806 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.199661 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z74f\" (UniqueName: \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.199867 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.199829 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.238779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.238744 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:23:21.238779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.238755 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:23:21.238972 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.238810 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:23:21.238972 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.238886 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:23:21.241170 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.241152 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b"] Nov 04 02:23:21.301496 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.301458 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.301496 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.301494 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z74f\" (UniqueName: \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.301691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.301530 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.302152 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.302133 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.303902 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.303885 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.362405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.362373 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z74f\" (UniqueName: \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\") pod \"maas-api-revoke-sub-be96d50e-bjh2b\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.447249 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.447172 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:21.822643 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:21.822612 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b"] Nov 04 02:23:21.825041 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:21.825013 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ed78e3_67b5_4ae4_9443_3d454a19e1f8.slice/crio-34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61 WatchSource:0}: Error finding container 34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61: Status 404 returned error can't find the container with id 34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61 Nov 04 02:23:22.320564 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:22.320527 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" event={"ID":"51ed78e3-67b5-4ae4-9443-3d454a19e1f8","Type":"ContainerStarted","Data":"002bfbff12cc3a3eeb88f787bccb6a7a95016cb2fa75b1b36293a216ff010fb1"} Nov 04 02:23:22.320994 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:22.320579 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" event={"ID":"51ed78e3-67b5-4ae4-9443-3d454a19e1f8","Type":"ContainerStarted","Data":"34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61"} Nov 04 02:23:22.437877 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:22.437780 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" podStartSLOduration=1.437764812 podStartE2EDuration="1.437764812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:22.435315451 +0000 UTC m=+980.131005683" watchObservedRunningTime="2026-11-04 02:23:22.437764812 +0000 UTC m=+980.133455040" Nov 04 02:23:23.328189 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:23.328154 6073 generic.go:356] "Generic (PLEG): container finished" podID="51ed78e3-67b5-4ae4-9443-3d454a19e1f8" containerID="002bfbff12cc3a3eeb88f787bccb6a7a95016cb2fa75b1b36293a216ff010fb1" exitCode=0 Nov 04 02:23:23.328653 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:23.328203 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" event={"ID":"51ed78e3-67b5-4ae4-9443-3d454a19e1f8","Type":"ContainerDied","Data":"002bfbff12cc3a3eeb88f787bccb6a7a95016cb2fa75b1b36293a216ff010fb1"} Nov 04 02:23:23.858937 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:23.858897 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:23:24.487313 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.487285 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:24.537685 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.537642 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\") pod \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " Nov 04 02:23:24.537844 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.537766 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\") pod \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " Nov 04 02:23:24.537917 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.537849 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\" (UniqueName: \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\") pod \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\" (UID: \"51ed78e3-67b5-4ae4-9443-3d454a19e1f8\") " Nov 04 02:23:24.538191 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.538166 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca" pod "51ed78e3-67b5-4ae4-9443-3d454a19e1f8" (UID: "51ed78e3-67b5-4ae4-9443-3d454a19e1f8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:24.538309 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.538296 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:24.540010 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.539979 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca" pod "51ed78e3-67b5-4ae4-9443-3d454a19e1f8" (UID: "51ed78e3-67b5-4ae4-9443-3d454a19e1f8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:24.540205 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.540188 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f" pod "51ed78e3-67b5-4ae4-9443-3d454a19e1f8" (UID: "51ed78e3-67b5-4ae4-9443-3d454a19e1f8"). InnerVolumeSpecName "kube-api-access-8z74f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:24.639193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.639074 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z74f\" (UniqueName: \"kubernetes.io/projected/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-kube-api-access-8z74f\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:24.639193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:24.639103 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51ed78e3-67b5-4ae4-9443-3d454a19e1f8-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:25.341279 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:25.341245 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" Nov 04 02:23:25.341510 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:25.341275 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be96d50e-bjh2b" event={"ID":"51ed78e3-67b5-4ae4-9443-3d454a19e1f8","Type":"ContainerDied","Data":"34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61"} Nov 04 02:23:25.341510 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:25.341360 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34a1632c7fce840d3fb36cfaea6a3f231a5a26b813f32bbaf7d4287a77751b61" Nov 04 02:23:25.474380 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:25.474345 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:23:28.685944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:28.685910 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:23:28.735359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:28.735312 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:23:34.705948 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.705906 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l"] Nov 04 02:23:34.706414 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.706399 6073 cpu_manager.go:414] "containerMap: removing container" podUID="51ed78e3-67b5-4ae4-9443-3d454a19e1f8" containerName="revoke-keys" Nov 04 02:23:34.706456 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.706415 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51ed78e3-67b5-4ae4-9443-3d454a19e1f8" containerName="revoke-keys" Nov 04 02:23:34.706522 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.706513 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="51ed78e3-67b5-4ae4-9443-3d454a19e1f8" containerName="revoke-keys" Nov 04 02:23:34.709953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.709939 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.793135 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.793102 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l"] Nov 04 02:23:34.822517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.822487 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:23:34.822691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.822523 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:23:34.822691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.822522 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:23:34.822691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.822582 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:23:34.874471 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.874441 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.874627 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.874524 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm5rr\" (UniqueName: \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.874683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.874638 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.975382 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.975252 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm5rr\" (UniqueName: \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.975573 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.975400 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.975573 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.975509 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.976117 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.976097 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:34.978016 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:34.977993 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:35.036194 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:35.036159 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm5rr\" (UniqueName: \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\") pod \"maas-api-revoke-sub-c70a9c61-wnj2l\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:35.322599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:35.322564 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:35.697628 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:35.697598 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l"] Nov 04 02:23:35.700012 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:35.699984 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3cf33e_9707_4ec2_8def_3d6a7a8c3cba.slice/crio-9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b WatchSource:0}: Error finding container 9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b: Status 404 returned error can't find the container with id 9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b Nov 04 02:23:36.413501 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:36.413390 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" event={"ID":"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba","Type":"ContainerStarted","Data":"4016c6f5fe2fca3571b05379117b7b8374763e9a695e28b1cbc5dad94b08b6ab"} Nov 04 02:23:36.414069 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:36.413507 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" event={"ID":"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba","Type":"ContainerStarted","Data":"9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b"} Nov 04 02:23:36.531907 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:36.531836 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" podStartSLOduration=2.5318160069999998 podStartE2EDuration="2.531816007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:36.52849203 +0000 UTC m=+994.224182231" watchObservedRunningTime="2026-11-04 02:23:36.531816007 +0000 UTC m=+994.227506210" Nov 04 02:23:37.421890 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:37.421855 6073 generic.go:356] "Generic (PLEG): container finished" podID="3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" containerID="4016c6f5fe2fca3571b05379117b7b8374763e9a695e28b1cbc5dad94b08b6ab" exitCode=0 Nov 04 02:23:37.422282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:37.421949 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" event={"ID":"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba","Type":"ContainerDied","Data":"4016c6f5fe2fca3571b05379117b7b8374763e9a695e28b1cbc5dad94b08b6ab"} Nov 04 02:23:38.069214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.069022 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:23:38.069582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.069323 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" containerID="cri-o://9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b" gracePeriod=30 Nov 04 02:23:38.075932 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.075902 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:23:38.076142 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.076115 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" containerID="cri-o://21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40" gracePeriod=30 Nov 04 02:23:38.568183 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.568158 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:38.720598 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.720518 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\") pod \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " Nov 04 02:23:38.720751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.720605 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\") pod \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " Nov 04 02:23:38.720751 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.720639 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\" (UniqueName: \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\") pod \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\" (UID: \"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba\") " Nov 04 02:23:38.720900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.720875 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca" pod "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" (UID: "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:38.722798 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.722773 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr" pod "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" (UID: "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba"). InnerVolumeSpecName "kube-api-access-pm5rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:38.722898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.722866 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca" pod "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" (UID: "3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:38.821722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.821675 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:38.821722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.821705 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:38.821722 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:38.821715 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm5rr\" (UniqueName: \"kubernetes.io/projected/3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba-kube-api-access-pm5rr\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:39.435084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:39.435047 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" event={"ID":"3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba","Type":"ContainerDied","Data":"9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b"} Nov 04 02:23:39.435273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:39.435097 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fe9e9bcc153e9b9aed7fe398d0834ad83c23bdda722aa99b81694e617b80b4b" Nov 04 02:23:39.435273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:39.435095 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70a9c61-wnj2l" Nov 04 02:23:40.495848 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.495805 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" podUID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" containerName="main" containerID="cri-o://1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813" gracePeriod=45 Nov 04 02:23:40.764170 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.764106 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:23:40.946258 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946225 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\" (UniqueName: \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946419 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946480 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946454 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946526 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946511 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\" (UniqueName: \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946568 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946548 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946596 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\") pod \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\" (UID: \"e6fcbfc6-eae2-4763-8c98-648130f8ff6f\") " Nov 04 02:23:40.946938 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946740 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:40.947065 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.946946 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:40.947130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.947065 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:40.947130 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.947110 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:40.947321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.947299 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:40.949474 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.949449 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:40.949474 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.949457 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:40.949615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:40.949453 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv" pod "e6fcbfc6-eae2-4763-8c98-648130f8ff6f" (UID: "e6fcbfc6-eae2-4763-8c98-648130f8ff6f"). InnerVolumeSpecName "kube-api-access-dpzjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:41.048529 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.048496 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:41.048529 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.048531 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpzjv\" (UniqueName: \"kubernetes.io/projected/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-kube-api-access-dpzjv\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:41.048747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.048547 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:41.048747 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.048558 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6fcbfc6-eae2-4763-8c98-648130f8ff6f-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:41.448640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.448547 6073 generic.go:356] "Generic (PLEG): container finished" podID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" containerID="1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813" exitCode=0 Nov 04 02:23:41.448640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.448590 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" Nov 04 02:23:41.448640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.448628 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" event={"ID":"e6fcbfc6-eae2-4763-8c98-648130f8ff6f","Type":"ContainerDied","Data":"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813"} Nov 04 02:23:41.448898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.448671 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7" event={"ID":"e6fcbfc6-eae2-4763-8c98-648130f8ff6f","Type":"ContainerDied","Data":"52a6b23003dc8cd8d8035fe33d0712f9a9325c791378bdc9eafb7b9416784049"} Nov 04 02:23:41.448898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.448689 6073 scope.go:122] "RemoveContainer" containerID="1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813" Nov 04 02:23:41.458910 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.458892 6073 scope.go:122] "RemoveContainer" containerID="1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813" Nov 04 02:23:41.459168 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:41.459147 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813\": container with ID starting with 1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813 not found: ID does not exist" containerID="1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813" Nov 04 02:23:41.459236 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.459176 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813"} err="failed to get container status \"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813\": rpc error: code = NotFound desc = could not find container \"1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813\": container with ID starting with 1919b79d10c1b819d2ab1dc48c3acd4171613e7f50120d9266d7dd8823f5c813 not found: ID does not exist" Nov 04 02:23:41.612095 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.612061 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:23:41.661380 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:41.661344 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/distinct-2-w6-9adedd-kserve-5b57694cf7-vl6l7"] Nov 04 02:23:42.425681 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.425583 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" path="/var/lib/kubelet/pods/e6fcbfc6-eae2-4763-8c98-648130f8ff6f/volumes" Nov 04 02:23:42.605906 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.605870 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw"] Nov 04 02:23:42.606349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606316 6073 cpu_manager.go:414] "containerMap: removing container" podUID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" containerName="main" Nov 04 02:23:42.606349 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606345 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" containerName="main" Nov 04 02:23:42.606435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606385 6073 cpu_manager.go:414] "containerMap: removing container" podUID="3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" containerName="revoke-keys" Nov 04 02:23:42.606435 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606393 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" containerName="revoke-keys" Nov 04 02:23:42.606503 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606480 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba" containerName="revoke-keys" Nov 04 02:23:42.606503 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.606493 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6fcbfc6-eae2-4763-8c98-648130f8ff6f" containerName="main" Nov 04 02:23:42.611169 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.611143 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.713367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.713241 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:23:42.713367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.713257 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:23:42.713367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.713295 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:23:42.713841 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.713379 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:23:42.716028 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.715971 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw"] Nov 04 02:23:42.767760 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.767720 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.767940 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.767792 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.767940 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.767851 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjf59\" (UniqueName: \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.868768 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.868734 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.868953 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.868774 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjf59\" (UniqueName: \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.869010 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.868978 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.869477 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.869453 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.871575 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.871548 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:42.929987 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:42.929948 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjf59\" (UniqueName: \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\") pod \"maas-api-revoke-sub-e24ba4ae-k44tw\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:43.224668 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.224434 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:43.345180 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.345103 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:23:43.379112 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.379090 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:23:43.462399 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.462364 6073 generic.go:356] "Generic (PLEG): container finished" podID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerID="9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b" exitCode=0 Nov 04 02:23:43.462586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.462423 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" Nov 04 02:23:43.462586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.462443 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" event={"ID":"caea0781-fac2-4f39-86c1-a250ccb6df4b","Type":"ContainerDied","Data":"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b"} Nov 04 02:23:43.462586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.462497 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp" event={"ID":"caea0781-fac2-4f39-86c1-a250ccb6df4b","Type":"ContainerDied","Data":"2c1ff49d94c060ee9df28d62fbe0eed9701074632912afb3821c2683e42c7ccd"} Nov 04 02:23:43.462586 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.462521 6073 scope.go:122] "RemoveContainer" containerID="9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b" Nov 04 02:23:43.464461 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.464426 6073 generic.go:356] "Generic (PLEG): container finished" podID="b784a230-c6a9-4829-a740-3be96fd836f5" containerID="21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40" exitCode=0 Nov 04 02:23:43.464600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.464476 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" Nov 04 02:23:43.464600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.464508 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerDied","Data":"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40"} Nov 04 02:23:43.464600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.464552 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc" event={"ID":"b784a230-c6a9-4829-a740-3be96fd836f5","Type":"ContainerDied","Data":"2fa93a65859430b5eea6e1334358c7696461091d62af144b040010d54577b474"} Nov 04 02:23:43.472498 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.472480 6073 scope.go:122] "RemoveContainer" containerID="9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b" Nov 04 02:23:43.472759 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:43.472742 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b\": container with ID starting with 9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b not found: ID does not exist" containerID="9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b" Nov 04 02:23:43.472804 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.472768 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b"} err="failed to get container status \"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b\": rpc error: code = NotFound desc = could not find container \"9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b\": container with ID starting with 9c518cacbd0e2e6a3f7cf7b7d3ff1ac6e00191f745f1bf6919cd76d6861f312b not found: ID does not exist" Nov 04 02:23:43.472804 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.472785 6073 scope.go:122] "RemoveContainer" containerID="21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40" Nov 04 02:23:43.475955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.475939 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476021 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.475968 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476021 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.475990 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476048 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476079 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476175 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476114 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.476226 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476171 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476226 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476207 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.476368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476216 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.476368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476309 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476378 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476410 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\") pod \"caea0781-fac2-4f39-86c1-a250ccb6df4b\" (UID: \"caea0781-fac2-4f39-86c1-a250ccb6df4b\") " Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476409 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476425 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476437 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.476484 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476450 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.476789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476514 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.476789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476551 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:43.476789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476564 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.476789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476750 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.476789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476762 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.477060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476872 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.477060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.476905 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477103 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477124 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477137 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477151 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477163 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caea0781-fac2-4f39-86c1-a250ccb6df4b-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477171 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477670 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477183 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477670 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477200 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.477670 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.477218 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.478959 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.478939 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "kube-api-access-4lsbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:43.479163 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.479137 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.479253 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.479231 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:23:43.479565 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.479545 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:43.479641 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.479581 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g" pod "caea0781-fac2-4f39-86c1-a250ccb6df4b" (UID: "caea0781-fac2-4f39-86c1-a250ccb6df4b"). InnerVolumeSpecName "kube-api-access-cls4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:43.483210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.483193 6073 scope.go:122] "RemoveContainer" containerID="168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0" Nov 04 02:23:43.494372 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.494348 6073 scope.go:122] "RemoveContainer" containerID="21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40" Nov 04 02:23:43.495028 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:43.495002 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40\": container with ID starting with 21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40 not found: ID does not exist" containerID="21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40" Nov 04 02:23:43.495144 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.495039 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40"} err="failed to get container status \"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40\": rpc error: code = NotFound desc = could not find container \"21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40\": container with ID starting with 21dcbde41b37c3bee5e54ed43c2da684e4e4a823a9bde644166c2302090fed40 not found: ID does not exist" Nov 04 02:23:43.495144 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.495061 6073 scope.go:122] "RemoveContainer" containerID="168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0" Nov 04 02:23:43.495481 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:43.495451 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0\": container with ID starting with 168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0 not found: ID does not exist" containerID="168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0" Nov 04 02:23:43.495594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.495490 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0"} err="failed to get container status \"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0\": rpc error: code = NotFound desc = could not find container \"168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0\": container with ID starting with 168490ade398c0e33889797569b153dafe54d75fcb45928e5f0c91b8a838ffb0 not found: ID does not exist" Nov 04 02:23:43.578520 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578487 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.578700 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578537 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.578700 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578579 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.578700 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578624 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\") pod \"b784a230-c6a9-4829-a740-3be96fd836f5\" (UID: \"b784a230-c6a9-4829-a740-3be96fd836f5\") " Nov 04 02:23:43.578930 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578899 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:43.579007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578935 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cls4g\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-kube-api-access-cls4g\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.579007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578953 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.579007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578963 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lsbb\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-kube-api-access-4lsbb\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.579007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578972 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.579007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.578980 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caea0781-fac2-4f39-86c1-a250ccb6df4b-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.580831 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.580807 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:43.580933 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.580914 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:23:43.580989 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.580971 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token" pod "b784a230-c6a9-4829-a740-3be96fd836f5" (UID: "b784a230-c6a9-4829-a740-3be96fd836f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:43.643365 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.636313 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw"] Nov 04 02:23:43.643365 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:43.639573 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51db9f6c_006e_4023_aca0_abfd9b2a6f24.slice/crio-2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6 WatchSource:0}: Error finding container 2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6: Status 404 returned error can't find the container with id 2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6 Nov 04 02:23:43.680898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.680857 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b784a230-c6a9-4829-a740-3be96fd836f5-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.680898 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.680895 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b784a230-c6a9-4829-a740-3be96fd836f5-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.681073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.680911 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b784a230-c6a9-4829-a740-3be96fd836f5-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.681073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.680925 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b784a230-c6a9-4829-a740-3be96fd836f5-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:43.707043 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.707000 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" podUID="2bf13801-e5e1-4fe5-931c-c913e086ae77" containerName="main" containerID="cri-o://cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792" gracePeriod=45 Nov 04 02:23:43.755626 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.755565 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" podUID="7288b257-9d95-469e-be28-b9b73bd81dde" containerName="main" containerID="cri-o://82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d" gracePeriod=45 Nov 04 02:23:43.917649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.916618 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:23:43.947192 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:43.947156 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-5fnlp"] Nov 04 02:23:44.068600 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.068565 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:23:44.077720 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.077693 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-68rvc"] Nov 04 02:23:44.101214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.101182 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:23:44.136306 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.136284 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:23:44.185508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185310 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.185508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185381 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.185508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185419 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\" (UniqueName: \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.185508 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185498 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.185883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185527 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.185883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185570 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\" (UniqueName: \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.185883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185599 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.185883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185732 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.185883 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185815 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.186125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.185957 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.186203 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186174 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\" (UniqueName: \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.186278 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186235 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.186367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186267 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.186367 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186308 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.186483 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186370 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\") pod \"7288b257-9d95-469e-be28-b9b73bd81dde\" (UID: \"7288b257-9d95-469e-be28-b9b73bd81dde\") " Nov 04 02:23:44.186538 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186483 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.186719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186602 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\") pod \"2bf13801-e5e1-4fe5-931c-c913e086ae77\" (UID: \"2bf13801-e5e1-4fe5-931c-c913e086ae77\") " Nov 04 02:23:44.186955 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.186921 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187073 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187093 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187107 6073 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-tmp-dir\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187119 6073 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-home\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187131 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.187202 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.187147 6073 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-model-cache\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.188625 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.188604 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "kube-api-access-d9kpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:44.189167 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.189130 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.189270 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.189199 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:44.189270 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.189215 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:44.189441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.189291 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5" pod "2bf13801-e5e1-4fe5-931c-c913e086ae77" (UID: "2bf13801-e5e1-4fe5-931c-c913e086ae77"). InnerVolumeSpecName "kube-api-access-skbj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:44.189857 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.189834 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm" pod "7288b257-9d95-469e-be28-b9b73bd81dde" (UID: "7288b257-9d95-469e-be28-b9b73bd81dde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:44.288220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288185 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf13801-e5e1-4fe5-931c-c913e086ae77-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.288220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288215 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9kpb\" (UniqueName: \"kubernetes.io/projected/7288b257-9d95-469e-be28-b9b73bd81dde-kube-api-access-d9kpb\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.288220 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288226 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skbj5\" (UniqueName: \"kubernetes.io/projected/2bf13801-e5e1-4fe5-931c-c913e086ae77-kube-api-access-skbj5\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.288488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288234 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf13801-e5e1-4fe5-931c-c913e086ae77-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.288488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288243 6073 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7288b257-9d95-469e-be28-b9b73bd81dde-dshm\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.288488 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.288251 6073 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7288b257-9d95-469e-be28-b9b73bd81dde-tls-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:44.426347 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.426302 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" path="/var/lib/kubelet/pods/b784a230-c6a9-4829-a740-3be96fd836f5/volumes" Nov 04 02:23:44.426796 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.426782 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" path="/var/lib/kubelet/pods/caea0781-fac2-4f39-86c1-a250ccb6df4b/volumes" Nov 04 02:23:44.478814 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.478769 6073 generic.go:356] "Generic (PLEG): container finished" podID="7288b257-9d95-469e-be28-b9b73bd81dde" containerID="82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d" exitCode=0 Nov 04 02:23:44.479042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.478846 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" Nov 04 02:23:44.479042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.478865 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" event={"ID":"7288b257-9d95-469e-be28-b9b73bd81dde","Type":"ContainerDied","Data":"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d"} Nov 04 02:23:44.479042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.478902 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g" event={"ID":"7288b257-9d95-469e-be28-b9b73bd81dde","Type":"ContainerDied","Data":"683845a7513c84eb27020ee70d216a4fca1b5ba8d3bcd21d515c5ecf7ac14348"} Nov 04 02:23:44.479042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.478920 6073 scope.go:122] "RemoveContainer" containerID="82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d" Nov 04 02:23:44.481199 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.481172 6073 generic.go:356] "Generic (PLEG): container finished" podID="2bf13801-e5e1-4fe5-931c-c913e086ae77" containerID="cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792" exitCode=0 Nov 04 02:23:44.481610 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.481504 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" event={"ID":"2bf13801-e5e1-4fe5-931c-c913e086ae77","Type":"ContainerDied","Data":"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792"} Nov 04 02:23:44.481610 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.481570 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" event={"ID":"2bf13801-e5e1-4fe5-931c-c913e086ae77","Type":"ContainerDied","Data":"6ceea81796d766689c134020df01eaa8e3d15b9da815a9094661686743b96325"} Nov 04 02:23:44.481767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.481655 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp" Nov 04 02:23:44.483499 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.483463 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" event={"ID":"51db9f6c-006e-4023-aca0-abfd9b2a6f24","Type":"ContainerStarted","Data":"fb6258932dff928630545acd704fa075f83eff719cf84ea58e71269ba0d27244"} Nov 04 02:23:44.483614 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.483515 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" event={"ID":"51db9f6c-006e-4023-aca0-abfd9b2a6f24","Type":"ContainerStarted","Data":"2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6"} Nov 04 02:23:44.489795 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.489779 6073 scope.go:122] "RemoveContainer" containerID="82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d" Nov 04 02:23:44.490138 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:44.490109 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d\": container with ID starting with 82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d not found: ID does not exist" containerID="82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d" Nov 04 02:23:44.490204 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.490149 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d"} err="failed to get container status \"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d\": rpc error: code = NotFound desc = could not find container \"82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d\": container with ID starting with 82b71d1ea64c0720e080bcb39fc3d5855721d05f5b20df8501ba3ad87242e79d not found: ID does not exist" Nov 04 02:23:44.490204 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.490174 6073 scope.go:122] "RemoveContainer" containerID="cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792" Nov 04 02:23:44.499874 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.499853 6073 scope.go:122] "RemoveContainer" containerID="cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792" Nov 04 02:23:44.500179 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:44.500152 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792\": container with ID starting with cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792 not found: ID does not exist" containerID="cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792" Nov 04 02:23:44.500242 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.500189 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792"} err="failed to get container status \"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792\": rpc error: code = NotFound desc = could not find container \"cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792\": container with ID starting with cfa95a06c468fc53586d28d1ff133134a75c26d37aca7dea195687451fbe9792 not found: ID does not exist" Nov 04 02:23:44.642149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.642117 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:23:44.646820 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.646792 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-w5-dd5d18-kserve-7b6d7454d8-gh6dp"] Nov 04 02:23:44.755145 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.755051 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:23:44.759451 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.759422 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/premium-w5-dd5d18-kserve-6b5c98679c-fn55g"] Nov 04 02:23:44.875260 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:44.875192 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" podStartSLOduration=2.875175978 podStartE2EDuration="2.875175978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:44.873048508 +0000 UTC m=+1002.568738709" watchObservedRunningTime="2026-11-04 02:23:44.875175978 +0000 UTC m=+1002.570866178" Nov 04 02:23:45.492763 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:45.492733 6073 generic.go:356] "Generic (PLEG): container finished" podID="51db9f6c-006e-4023-aca0-abfd9b2a6f24" containerID="fb6258932dff928630545acd704fa075f83eff719cf84ea58e71269ba0d27244" exitCode=0 Nov 04 02:23:45.492951 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:45.492820 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" event={"ID":"51db9f6c-006e-4023-aca0-abfd9b2a6f24","Type":"ContainerDied","Data":"fb6258932dff928630545acd704fa075f83eff719cf84ea58e71269ba0d27244"} Nov 04 02:23:46.428657 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.428624 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2bf13801-e5e1-4fe5-931c-c913e086ae77" path="/var/lib/kubelet/pods/2bf13801-e5e1-4fe5-931c-c913e086ae77/volumes" Nov 04 02:23:46.429191 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.429015 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7288b257-9d95-469e-be28-b9b73bd81dde" path="/var/lib/kubelet/pods/7288b257-9d95-469e-be28-b9b73bd81dde/volumes" Nov 04 02:23:46.679231 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.679174 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:46.713234 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.713205 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\") pod \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " Nov 04 02:23:46.713386 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.713249 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\") pod \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " Nov 04 02:23:46.713441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.713424 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\" (UniqueName: \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\") pod \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\" (UID: \"51db9f6c-006e-4023-aca0-abfd9b2a6f24\") " Nov 04 02:23:46.715405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.713866 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca" pod "51db9f6c-006e-4023-aca0-abfd9b2a6f24" (UID: "51db9f6c-006e-4023-aca0-abfd9b2a6f24"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:46.715405 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.714242 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:46.716690 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.716659 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59" pod "51db9f6c-006e-4023-aca0-abfd9b2a6f24" (UID: "51db9f6c-006e-4023-aca0-abfd9b2a6f24"). InnerVolumeSpecName "kube-api-access-kjf59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:46.721667 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.721643 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca" pod "51db9f6c-006e-4023-aca0-abfd9b2a6f24" (UID: "51db9f6c-006e-4023-aca0-abfd9b2a6f24"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:23:46.815277 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.815239 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjf59\" (UniqueName: \"kubernetes.io/projected/51db9f6c-006e-4023-aca0-abfd9b2a6f24-kube-api-access-kjf59\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:46.815277 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:46.815279 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51db9f6c-006e-4023-aca0-abfd9b2a6f24-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:47.517397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:47.517369 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" Nov 04 02:23:47.517397 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:47.517378 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e24ba4ae-k44tw" event={"ID":"51db9f6c-006e-4023-aca0-abfd9b2a6f24","Type":"ContainerDied","Data":"2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6"} Nov 04 02:23:47.517926 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:47.517432 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e0d4136ad9c2c7842ff436a7c8b06efde8e090e11554d078e5cb5b8542493d6" Nov 04 02:23:51.751820 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.751781 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:23:51.752244 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752232 6073 cpu_manager.go:414] "containerMap: removing container" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752245 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752258 6073 cpu_manager.go:414] "containerMap: removing container" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752263 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752282 6073 cpu_manager.go:414] "containerMap: removing container" podUID="51db9f6c-006e-4023-aca0-abfd9b2a6f24" containerName="revoke-keys" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752288 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51db9f6c-006e-4023-aca0-abfd9b2a6f24" containerName="revoke-keys" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752300 6073 cpu_manager.go:414] "containerMap: removing container" podUID="2bf13801-e5e1-4fe5-931c-c913e086ae77" containerName="main" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752304 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2bf13801-e5e1-4fe5-931c-c913e086ae77" containerName="main" Nov 04 02:23:51.752314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752317 6073 cpu_manager.go:414] "containerMap: removing container" podUID="7288b257-9d95-469e-be28-b9b73bd81dde" containerName="main" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752323 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7288b257-9d95-469e-be28-b9b73bd81dde" containerName="main" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752412 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="7288b257-9d95-469e-be28-b9b73bd81dde" containerName="main" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752421 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752428 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="caea0781-fac2-4f39-86c1-a250ccb6df4b" containerName="istio-proxy" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752435 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752441 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="51db9f6c-006e-4023-aca0-abfd9b2a6f24" containerName="revoke-keys" Nov 04 02:23:51.752762 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.752447 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="2bf13801-e5e1-4fe5-931c-c913e086ae77" containerName="main" Nov 04 02:23:51.755761 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.755745 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:51.809899 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.809861 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-9adedd\"" Nov 04 02:23:51.812390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.812364 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:23:51.861598 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.861571 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:23:51.862033 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.862020 6073 cpu_manager.go:414] "containerMap: removing container" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.862091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.862034 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b784a230-c6a9-4829-a740-3be96fd836f5" containerName="istio-proxy" Nov 04 02:23:51.865258 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.865232 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:51.869412 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.869382 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:51.869544 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.869488 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pdwr\" (UniqueName: \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:51.898769 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.898732 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:23:51.917149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.917122 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-9adedd-dockercfg-rv48h\"" Nov 04 02:23:51.970519 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.970489 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:51.970676 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.970527 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:51.970676 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.970557 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmjx\" (UniqueName: \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:51.970676 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.970670 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pdwr\" (UniqueName: \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:51.971228 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:51.971206 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:52.031879 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.031844 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pdwr\" (UniqueName: \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:52.069869 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.069835 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:52.071298 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.071276 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:52.071375 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.071319 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmjx\" (UniqueName: \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:52.071950 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.071931 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:52.141769 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.141688 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmjx\" (UniqueName: \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:52.178649 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.178618 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:52.446240 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.446215 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:23:52.448272 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:52.448243 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085ce452_b573_4450_8617_ec06a0eb88e0.slice/crio-1659b32c9c77de7df7d5c90e9bc69b1371af1ec553ce43b8a902a425ad7282a9 WatchSource:0}: Error finding container 1659b32c9c77de7df7d5c90e9bc69b1371af1ec553ce43b8a902a425ad7282a9: Status 404 returned error can't find the container with id 1659b32c9c77de7df7d5c90e9bc69b1371af1ec553ce43b8a902a425ad7282a9 Nov 04 02:23:52.549046 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.548996 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" event={"ID":"085ce452-b573-4450-8617-ec06a0eb88e0","Type":"ContainerStarted","Data":"1659b32c9c77de7df7d5c90e9bc69b1371af1ec553ce43b8a902a425ad7282a9"} Nov 04 02:23:52.552279 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:52.552259 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:23:52.553900 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:23:52.553873 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bf0399_7d90_40fc_85f6_7310ef9f9610.slice/crio-fa2c07d6439894a8f6eb59ad1f0f34f388e3c5eba173fd15fa38d4184eaa293b WatchSource:0}: Error finding container fa2c07d6439894a8f6eb59ad1f0f34f388e3c5eba173fd15fa38d4184eaa293b: Status 404 returned error can't find the container with id fa2c07d6439894a8f6eb59ad1f0f34f388e3c5eba173fd15fa38d4184eaa293b Nov 04 02:23:53.556279 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:53.556240 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" event={"ID":"f6bf0399-7d90-40fc-85f6-7310ef9f9610","Type":"ContainerStarted","Data":"fa2c07d6439894a8f6eb59ad1f0f34f388e3c5eba173fd15fa38d4184eaa293b"} Nov 04 02:23:53.879935 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:23:53.879850 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:23:56.588611 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.588572 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" event={"ID":"085ce452-b573-4450-8617-ec06a0eb88e0","Type":"ContainerStarted","Data":"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81"} Nov 04 02:23:56.589045 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.588642 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:23:56.590210 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.590182 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" event={"ID":"f6bf0399-7d90-40fc-85f6-7310ef9f9610","Type":"ContainerStarted","Data":"fc52c2d1850fa39c7c65c1ed96cc4692e3727df6a024e18369ae256c48359c0e"} Nov 04 02:23:56.590320 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.590272 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:23:56.709010 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.708944 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" podStartSLOduration=2.003570774 podStartE2EDuration="5.708924802s" totalImagesPullingTime="3.705354028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:56.703136762 +0000 UTC m=+1014.398826975" watchObservedRunningTime="2026-11-04 02:23:56.708924802 +0000 UTC m=+1014.404615006" Nov 04 02:23:56.821346 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:23:56.821284 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" podStartSLOduration=2.213093098 podStartE2EDuration="5.821271833s" totalImagesPullingTime="3.608178735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:56.819380511 +0000 UTC m=+1014.515070711" watchObservedRunningTime="2026-11-04 02:23:56.821271833 +0000 UTC m=+1014.516962029" Nov 04 02:24:02.698154 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:02.698121 6073 scope.go:122] "RemoveContainer" containerID="7f4ff8ca836024b2da144766a561869e4a4cb59c5fd17747cf4ee6f33214cce1" Nov 04 02:24:06.166382 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.166350 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt"] Nov 04 02:24:06.171808 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.171786 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.225383 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.225128 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 02:24:06.225585 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.225189 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bvs2z\"" Nov 04 02:24:06.256896 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.256865 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkfjn\" (UniqueName: \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.257060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.257007 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.257125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.257039 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.274974 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.274947 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:24:06.274974 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.274959 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:24:06.277308 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.277291 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt"] Nov 04 02:24:06.357801 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.357762 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.358001 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.357897 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkfjn\" (UniqueName: \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.358091 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.358072 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.358467 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.358446 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.361664 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.361645 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.420483 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.420408 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkfjn\" (UniqueName: \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\") pod \"maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.484683 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.484648 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:06.860441 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:06.860409 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt"] Nov 04 02:24:06.862492 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:24:06.862463 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b87197_3eec_47f0_a796_e3de2cc3223c.slice/crio-c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef WatchSource:0}: Error finding container c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef: Status 404 returned error can't find the container with id c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef Nov 04 02:24:07.599517 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:07.599488 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:24:07.599909 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:07.599560 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:24:07.658034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:07.657995 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" event={"ID":"38b87197-3eec-47f0-a796-e3de2cc3223c","Type":"ContainerStarted","Data":"5d1cfba2164fa97cb5a3b68badb961f0fb4a5f5f235123807f09bb26b2a3c9e8"} Nov 04 02:24:07.658178 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:07.658048 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" event={"ID":"38b87197-3eec-47f0-a796-e3de2cc3223c","Type":"ContainerStarted","Data":"c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef"} Nov 04 02:24:07.948322 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:07.948217 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" podStartSLOduration=1.948200778 podStartE2EDuration="1.948200778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:24:07.947027982 +0000 UTC m=+1025.642718177" watchObservedRunningTime="2026-11-04 02:24:07.948200778 +0000 UTC m=+1025.643890979" Nov 04 02:24:08.666060 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.666020 6073 generic.go:356] "Generic (PLEG): container finished" podID="38b87197-3eec-47f0-a796-e3de2cc3223c" containerID="5d1cfba2164fa97cb5a3b68badb961f0fb4a5f5f235123807f09bb26b2a3c9e8" exitCode=0 Nov 04 02:24:08.666596 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.666303 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" event={"ID":"38b87197-3eec-47f0-a796-e3de2cc3223c","Type":"ContainerDied","Data":"5d1cfba2164fa97cb5a3b68badb961f0fb4a5f5f235123807f09bb26b2a3c9e8"} Nov 04 02:24:08.718273 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.718241 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:08.722188 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.722164 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.784079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.784036 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:08.788408 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788377 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788425 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788490 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788550 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788582 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788570 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788603 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788647 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jw6d\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788695 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.788860 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.788753 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.889739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889708 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889744 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889784 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889830 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889870 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889900 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889941 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jw6d\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890003 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.889982 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890041 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890224 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890433 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890279 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890611 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890490 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890611 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890542 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.890719 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.890684 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.892358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.892322 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.892522 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.892506 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.956929 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.956858 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:08.958636 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:08.958611 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jw6d\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:09.036501 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.036439 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:09.479131 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.479104 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:09.480655 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:24:09.480622 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae6d5cb_03e3_4738_929a_453217c8688a.slice/crio-51164eff57ed763dd2687b7b130a3d11d61a5bed1d61b065414932243378c4fc WatchSource:0}: Error finding container 51164eff57ed763dd2687b7b130a3d11d61a5bed1d61b065414932243378c4fc: Status 404 returned error can't find the container with id 51164eff57ed763dd2687b7b130a3d11d61a5bed1d61b065414932243378c4fc Nov 04 02:24:09.483767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.483185 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:24:09.483767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.483275 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:24:09.483767 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.483320 6073 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"} Nov 04 02:24:09.683553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.683391 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" event={"ID":"3ae6d5cb-03e3-4738-929a-453217c8688a","Type":"ContainerStarted","Data":"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb"} Nov 04 02:24:09.683553 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.683498 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" event={"ID":"3ae6d5cb-03e3-4738-929a-453217c8688a","Type":"ContainerStarted","Data":"51164eff57ed763dd2687b7b130a3d11d61a5bed1d61b065414932243378c4fc"} Nov 04 02:24:09.809401 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.809318 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podStartSLOduration=1.809296905 podStartE2EDuration="1.809296905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:24:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:24:09.804382487 +0000 UTC m=+1027.500072689" watchObservedRunningTime="2026-11-04 02:24:09.809296905 +0000 UTC m=+1027.504987106" Nov 04 02:24:09.843317 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.843019 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:09.902466 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.902437 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\") pod \"38b87197-3eec-47f0-a796-e3de2cc3223c\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " Nov 04 02:24:09.902633 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.902607 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\" (UniqueName: \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\") pod \"38b87197-3eec-47f0-a796-e3de2cc3223c\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " Nov 04 02:24:09.902773 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.902655 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\") pod \"38b87197-3eec-47f0-a796-e3de2cc3223c\" (UID: \"38b87197-3eec-47f0-a796-e3de2cc3223c\") " Nov 04 02:24:09.903834 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.903724 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca" pod "38b87197-3eec-47f0-a796-e3de2cc3223c" (UID: "38b87197-3eec-47f0-a796-e3de2cc3223c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:09.906017 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.905982 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca" pod "38b87197-3eec-47f0-a796-e3de2cc3223c" (UID: "38b87197-3eec-47f0-a796-e3de2cc3223c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:24:09.907198 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:09.907172 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn" pod "38b87197-3eec-47f0-a796-e3de2cc3223c" (UID: "38b87197-3eec-47f0-a796-e3de2cc3223c"). InnerVolumeSpecName "kube-api-access-vkfjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:10.006402 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.004431 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkfjn\" (UniqueName: \"kubernetes.io/projected/38b87197-3eec-47f0-a796-e3de2cc3223c-kube-api-access-vkfjn\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:10.006402 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.004479 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-xks-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:10.006402 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.004494 6073 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38b87197-3eec-47f0-a796-e3de2cc3223c-maas-api-openshift-ca\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:10.038991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.038869 6073 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:10.040588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.040564 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:10.040706 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.040622 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:10.690468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.690385 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" Nov 04 02:24:10.690468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.690384 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt" event={"ID":"38b87197-3eec-47f0-a796-e3de2cc3223c","Type":"ContainerDied","Data":"c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef"} Nov 04 02:24:10.690468 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:10.690450 6073 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c827190fe1d7315f25798ad629443ee29a4824a08cca1121f48462a55e1faeef" Nov 04 02:24:11.039149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:11.039110 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:11.039382 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:11.039171 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:12.050359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:12.041715 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:12.050359 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:12.041773 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:13.039983 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:13.039208 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:13.039983 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:13.039269 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:14.038892 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:14.038852 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:14.039295 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:14.038914 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:15.038991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.038959 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 02:24:15.039440 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.039014 6073 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 02:24:15.980318 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.980285 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:24:15.980559 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.980536 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" containerID="cri-o://0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8" gracePeriod=30 Nov 04 02:24:15.993721 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.993693 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:15.993962 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:15.993936 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" containerID="cri-o://46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb" gracePeriod=30 Nov 04 02:24:16.010495 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:16.010465 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:24:16.010749 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:16.010721 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" containerID="cri-o://1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a" gracePeriod=30 Nov 04 02:24:16.629306 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:16.629258 6073 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 02:24:16.629830 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:16.629319 6073 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 02:24:19.621284 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:19.621250 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:24:19.739096 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:19.739054 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:24:21.255550 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.255484 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:24:21.288760 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.288738 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:24:21.402780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402744 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.402780 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402782 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402806 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402834 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402877 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402896 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402935 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.402963 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403070 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403109 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403138 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403216 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403248 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\") pod \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\" (UID: \"c6e94ae4-8c5a-472b-b107-48e929c3caa2\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403300 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403371 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403400 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403439 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403428 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403885 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403467 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\") pod \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\" (UID: \"b37ebe37-80b0-408c-b2e8-451e02f0be6e\") " Nov 04 02:24:21.403885 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403493 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.403885 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403527 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.403885 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.403720 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.404082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404002 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.404082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404023 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.404082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404040 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.404082 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404062 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.404276 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404124 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.404276 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404142 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.404659 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404524 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:21.404851 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.404803 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.405106 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.405080 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.406861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.405947 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:21.406861 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.406664 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:24:21.408054 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.408028 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:21.408556 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.408533 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.408833 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.408730 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:24:21.408833 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.408783 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:21.409099 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.409073 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:21.409622 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.409597 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n" pod "c6e94ae4-8c5a-472b-b107-48e929c3caa2" (UID: "c6e94ae4-8c5a-472b-b107-48e929c3caa2"). InnerVolumeSpecName "kube-api-access-htl8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:21.409744 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.409728 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59" pod "b37ebe37-80b0-408c-b2e8-451e02f0be6e" (UID: "b37ebe37-80b0-408c-b2e8-451e02f0be6e"). InnerVolumeSpecName "kube-api-access-vdq59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:21.504680 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504643 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504699 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504715 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504728 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504739 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504751 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504763 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504774 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504786 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504797 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504810 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdq59\" (UniqueName: \"kubernetes.io/projected/b37ebe37-80b0-408c-b2e8-451e02f0be6e-kube-api-access-vdq59\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504824 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b37ebe37-80b0-408c-b2e8-451e02f0be6e-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504836 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htl8n\" (UniqueName: \"kubernetes.io/projected/c6e94ae4-8c5a-472b-b107-48e929c3caa2-kube-api-access-htl8n\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504848 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b37ebe37-80b0-408c-b2e8-451e02f0be6e-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.504876 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.504860 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e94ae4-8c5a-472b-b107-48e929c3caa2-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:21.789029 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.788991 6073 generic.go:356] "Generic (PLEG): container finished" podID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerID="0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8" exitCode=0 Nov 04 02:24:21.789029 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.789032 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" Nov 04 02:24:21.789282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.789075 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" event={"ID":"b37ebe37-80b0-408c-b2e8-451e02f0be6e","Type":"ContainerDied","Data":"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8"} Nov 04 02:24:21.789282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.789123 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4" event={"ID":"b37ebe37-80b0-408c-b2e8-451e02f0be6e","Type":"ContainerDied","Data":"fdf2d44b88a2aa26366b56aef850a155f78419eb9c00618d2262c31f87111d32"} Nov 04 02:24:21.789282 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.789143 6073 scope.go:122] "RemoveContainer" containerID="0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8" Nov 04 02:24:21.791153 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.791122 6073 generic.go:356] "Generic (PLEG): container finished" podID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerID="1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a" exitCode=0 Nov 04 02:24:21.791153 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.791161 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" Nov 04 02:24:21.791323 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.791157 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerDied","Data":"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a"} Nov 04 02:24:21.791323 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.791215 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w" event={"ID":"c6e94ae4-8c5a-472b-b107-48e929c3caa2","Type":"ContainerDied","Data":"91952b225ccdb29daf638be18893ca860cfb9f494b04b08853a2a474aba6d0f6"} Nov 04 02:24:21.799122 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.799109 6073 scope.go:122] "RemoveContainer" containerID="0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8" Nov 04 02:24:21.799510 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:21.799491 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8\": container with ID starting with 0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8 not found: ID does not exist" containerID="0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8" Nov 04 02:24:21.799594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.799522 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8"} err="failed to get container status \"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8\": rpc error: code = NotFound desc = could not find container \"0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8\": container with ID starting with 0b74741dbaca43da6b998eec25ffaa71c386add47aefd60440e8b264e813b3b8 not found: ID does not exist" Nov 04 02:24:21.799594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.799545 6073 scope.go:122] "RemoveContainer" containerID="1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a" Nov 04 02:24:21.808552 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.808532 6073 scope.go:122] "RemoveContainer" containerID="5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24" Nov 04 02:24:21.817107 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.817091 6073 scope.go:122] "RemoveContainer" containerID="1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a" Nov 04 02:24:21.817424 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:21.817405 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a\": container with ID starting with 1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a not found: ID does not exist" containerID="1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a" Nov 04 02:24:21.817494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.817430 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a"} err="failed to get container status \"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a\": rpc error: code = NotFound desc = could not find container \"1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a\": container with ID starting with 1ccc48625b9cccabc0124422672a0c522454a679c4db128945daaf434d9d011a not found: ID does not exist" Nov 04 02:24:21.817494 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.817447 6073 scope.go:122] "RemoveContainer" containerID="5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24" Nov 04 02:24:21.817724 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:21.817701 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24\": container with ID starting with 5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24 not found: ID does not exist" containerID="5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24" Nov 04 02:24:21.817786 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.817733 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24"} err="failed to get container status \"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24\": rpc error: code = NotFound desc = could not find container \"5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24\": container with ID starting with 5fc88937bd6e9837e0d3707a4f7df5128ec19d126c15ea744b1209da7dd2fb24 not found: ID does not exist" Nov 04 02:24:21.955376 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.955321 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:24:21.958738 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:21.958713 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-cjp8w"] Nov 04 02:24:22.067034 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.067004 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:24:22.071314 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.071292 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-rzxb4"] Nov 04 02:24:22.242859 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.242831 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:22.413865 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.413783 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.413866 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.413918 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.413954 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414005 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414060 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414168 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414195 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414224 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414281 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:22.414316 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414299 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\") pod \"3ae6d5cb-03e3-4738-929a-453217c8688a\" (UID: \"3ae6d5cb-03e3-4738-929a-453217c8688a\") " Nov 04 02:24:22.414873 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414472 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:22.414873 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414482 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:22.414941 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414899 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:22.415007 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.414986 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.415042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.415015 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.415042 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.415030 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.415102 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.415045 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.416226 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.416204 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:24:22.416317 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.416226 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:22.416954 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.416932 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:22.417023 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.417007 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d" pod "3ae6d5cb-03e3-4738-929a-453217c8688a" (UID: "3ae6d5cb-03e3-4738-929a-453217c8688a"). InnerVolumeSpecName "kube-api-access-2jw6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:22.427669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.427641 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" path="/var/lib/kubelet/pods/b37ebe37-80b0-408c-b2e8-451e02f0be6e/volumes" Nov 04 02:24:22.428123 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.428109 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" path="/var/lib/kubelet/pods/c6e94ae4-8c5a-472b-b107-48e929c3caa2/volumes" Nov 04 02:24:22.516437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.516406 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.516437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.516431 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jw6d\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-kube-api-access-2jw6d\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.516437 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.516442 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.516669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.516452 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae6d5cb-03e3-4738-929a-453217c8688a-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.516669 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.516460 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae6d5cb-03e3-4738-929a-453217c8688a-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:22.798942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.798904 6073 generic.go:356] "Generic (PLEG): container finished" podID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerID="46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb" exitCode=0 Nov 04 02:24:22.799125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.798955 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" Nov 04 02:24:22.799125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.798990 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" event={"ID":"3ae6d5cb-03e3-4738-929a-453217c8688a","Type":"ContainerDied","Data":"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb"} Nov 04 02:24:22.799125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.799045 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m" event={"ID":"3ae6d5cb-03e3-4738-929a-453217c8688a","Type":"ContainerDied","Data":"51164eff57ed763dd2687b7b130a3d11d61a5bed1d61b065414932243378c4fc"} Nov 04 02:24:22.799125 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.799078 6073 scope.go:122] "RemoveContainer" containerID="46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb" Nov 04 02:24:22.808900 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.808881 6073 scope.go:122] "RemoveContainer" containerID="46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb" Nov 04 02:24:22.809145 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:22.809125 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb\": container with ID starting with 46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb not found: ID does not exist" containerID="46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb" Nov 04 02:24:22.809193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.809160 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb"} err="failed to get container status \"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb\": rpc error: code = NotFound desc = could not find container \"46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb\": container with ID starting with 46f1a8bea9ba8037d8c6443ae53ee716e1ad2c35b84ac245b783155a0d7173bb not found: ID does not exist" Nov 04 02:24:22.963509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.963474 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:22.967568 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:22.967538 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-q5b8m"] Nov 04 02:24:23.858641 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:23.858602 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:24:24.425348 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.425296 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" path="/var/lib/kubelet/pods/3ae6d5cb-03e3-4738-929a-453217c8688a/volumes" Nov 04 02:24:24.621676 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.621589 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" podUID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" containerName="payload-processing" containerID="cri-o://fc52c2d1850fa39c7c65c1ed96cc4692e3727df6a024e18369ae256c48359c0e" gracePeriod=25 Nov 04 02:24:24.739519 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.739438 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" podUID="085ce452-b573-4450-8617-ec06a0eb88e0" containerName="payload-pre-processing" containerID="cri-o://2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81" gracePeriod=25 Nov 04 02:24:24.819615 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.819580 6073 generic.go:356] "Generic (PLEG): container finished" podID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" containerID="fc52c2d1850fa39c7c65c1ed96cc4692e3727df6a024e18369ae256c48359c0e" exitCode=0 Nov 04 02:24:24.820215 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.819742 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" event={"ID":"f6bf0399-7d90-40fc-85f6-7310ef9f9610","Type":"ContainerDied","Data":"fc52c2d1850fa39c7c65c1ed96cc4692e3727df6a024e18369ae256c48359c0e"} Nov 04 02:24:24.877390 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.877365 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:24:24.989901 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:24.989839 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:24:25.041606 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.041567 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\" (UniqueName: \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\") pod \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " Nov 04 02:24:25.041778 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.041702 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\") pod \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\" (UID: \"f6bf0399-7d90-40fc-85f6-7310ef9f9610\") " Nov 04 02:24:25.042079 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.042053 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume" pod "f6bf0399-7d90-40fc-85f6-7310ef9f9610" (UID: "f6bf0399-7d90-40fc-85f6-7310ef9f9610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:25.043820 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.043801 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx" pod "f6bf0399-7d90-40fc-85f6-7310ef9f9610" (UID: "f6bf0399-7d90-40fc-85f6-7310ef9f9610"). InnerVolumeSpecName "kube-api-access-9zmjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:25.142877 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.142848 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\") pod \"085ce452-b573-4450-8617-ec06a0eb88e0\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " Nov 04 02:24:25.143051 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.143018 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\" (UniqueName: \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\") pod \"085ce452-b573-4450-8617-ec06a0eb88e0\" (UID: \"085ce452-b573-4450-8617-ec06a0eb88e0\") " Nov 04 02:24:25.143275 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.143254 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume" pod "085ce452-b573-4450-8617-ec06a0eb88e0" (UID: "085ce452-b573-4450-8617-ec06a0eb88e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:25.143392 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.143378 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zmjx\" (UniqueName: \"kubernetes.io/projected/f6bf0399-7d90-40fc-85f6-7310ef9f9610-kube-api-access-9zmjx\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:25.143432 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.143395 6073 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/085ce452-b573-4450-8617-ec06a0eb88e0-plugins-config-volume\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:25.143432 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.143404 6073 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bf0399-7d90-40fc-85f6-7310ef9f9610-plugins-config-volume\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:25.145204 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.145183 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr" pod "085ce452-b573-4450-8617-ec06a0eb88e0" (UID: "085ce452-b573-4450-8617-ec06a0eb88e0"). InnerVolumeSpecName "kube-api-access-7pdwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:25.244906 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.244827 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pdwr\" (UniqueName: \"kubernetes.io/projected/085ce452-b573-4450-8617-ec06a0eb88e0-kube-api-access-7pdwr\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:25.827012 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.826986 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" Nov 04 02:24:25.827214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.827025 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs" event={"ID":"f6bf0399-7d90-40fc-85f6-7310ef9f9610","Type":"ContainerDied","Data":"fa2c07d6439894a8f6eb59ad1f0f34f388e3c5eba173fd15fa38d4184eaa293b"} Nov 04 02:24:25.827214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.827086 6073 scope.go:122] "RemoveContainer" containerID="fc52c2d1850fa39c7c65c1ed96cc4692e3727df6a024e18369ae256c48359c0e" Nov 04 02:24:25.828541 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.828472 6073 generic.go:356] "Generic (PLEG): container finished" podID="085ce452-b573-4450-8617-ec06a0eb88e0" containerID="2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81" exitCode=0 Nov 04 02:24:25.828541 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.828506 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" event={"ID":"085ce452-b573-4450-8617-ec06a0eb88e0","Type":"ContainerDied","Data":"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81"} Nov 04 02:24:25.828637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.828537 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" Nov 04 02:24:25.828637 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.828544 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6" event={"ID":"085ce452-b573-4450-8617-ec06a0eb88e0","Type":"ContainerDied","Data":"1659b32c9c77de7df7d5c90e9bc69b1371af1ec553ce43b8a902a425ad7282a9"} Nov 04 02:24:25.842323 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.842308 6073 scope.go:122] "RemoveContainer" containerID="2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81" Nov 04 02:24:25.850707 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.850689 6073 scope.go:122] "RemoveContainer" containerID="2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81" Nov 04 02:24:25.850946 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:25.850929 6073 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81\": container with ID starting with 2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81 not found: ID does not exist" containerID="2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81" Nov 04 02:24:25.851004 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.850954 6073 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81"} err="failed to get container status \"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81\": rpc error: code = NotFound desc = could not find container \"2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81\": container with ID starting with 2729638e3c886f1fe7ba1441aa65a3a4f88ccedccf9b1de857db259ced3e9c81 not found: ID does not exist" Nov 04 02:24:25.991207 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:25.991131 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:24:26.040324 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:26.040296 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-zr9fs"] Nov 04 02:24:26.148925 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:26.148892 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:24:26.155509 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:26.155486 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-wq4j6"] Nov 04 02:24:26.424946 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:26.424910 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="085ce452-b573-4450-8617-ec06a0eb88e0" path="/var/lib/kubelet/pods/085ce452-b573-4450-8617-ec06a0eb88e0/volumes" Nov 04 02:24:26.425342 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:26.425314 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" path="/var/lib/kubelet/pods/f6bf0399-7d90-40fc-85f6-7310ef9f9610/volumes" Nov 04 02:24:27.463964 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:27.463922 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt"] Nov 04 02:24:27.513806 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:27.513774 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-9adedd-lvckt"] Nov 04 02:24:27.735835 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:27.735738 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:24:27.736056 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:27.736027 6073 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" containerID="cri-o://e91355bda496d71200feab5b9663ee71bc833ff8af36f1d69be828ba91eb969a" gracePeriod=30 Nov 04 02:24:28.425694 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:28.425664 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38b87197-3eec-47f0-a796-e3de2cc3223c" path="/var/lib/kubelet/pods/38b87197-3eec-47f0-a796-e3de2cc3223c/volumes" Nov 04 02:24:32.877491 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:32.877453 6073 generic.go:356] "Generic (PLEG): container finished" podID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerID="e91355bda496d71200feab5b9663ee71bc833ff8af36f1d69be828ba91eb969a" exitCode=0 Nov 04 02:24:32.878027 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:32.877501 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerDied","Data":"e91355bda496d71200feab5b9663ee71bc833ff8af36f1d69be828ba91eb969a"} Nov 04 02:24:32.878027 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:32.877550 6073 scope.go:122] "RemoveContainer" containerID="668f60e7790dff9dace0f733a20d7bde3a0a53ee4a2174352e0fe97efce23928" Nov 04 02:24:33.003933 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.003904 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:24:33.013887 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.013841 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.013965 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.013906 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.013965 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.013941 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.013965 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.013960 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.013990 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014030 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014073 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014047 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014071 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014214 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014125 6073 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\") pod \"51be9b55-dcd8-447a-83e3-303454b27fc4\" (UID: \"51be9b55-dcd8-447a-83e3-303454b27fc4\") " Nov 04 02:24:33.014321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014248 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:33.014321 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014260 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:33.014470 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014371 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:33.014527 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014488 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:33.014642 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014621 6073 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-data\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.014745 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014644 6073 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-certs\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.014745 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014644 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:24:33.014745 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014659 6073 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-workload-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.014745 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.014704 6073 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-credential-socket\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.016464 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.016440 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "kube-api-access-njk4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:33.016572 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.016551 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:24:33.016673 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.016649 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:24:33.016733 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.016655 6073 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token" pod "51be9b55-dcd8-447a-83e3-303454b27fc4" (UID: "51be9b55-dcd8-447a-83e3-303454b27fc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:24:33.116015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.115981 6073 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njk4j\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-kube-api-access-njk4j\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.116015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.116007 6073 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-token\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.116015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.116016 6073 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-envoy\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.116015 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.116024 6073 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51be9b55-dcd8-447a-83e3-303454b27fc4-istiod-ca-cert\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.116280 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.116032 6073 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51be9b55-dcd8-447a-83e3-303454b27fc4-istio-podinfo\") on node \"ip-10-0-138-45.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:24:33.890164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.890127 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" event={"ID":"51be9b55-dcd8-447a-83e3-303454b27fc4","Type":"ContainerDied","Data":"590841a18380819e68ecb2ea78ea4ec3a43104e70a851994b05f7a107c3ecedf"} Nov 04 02:24:33.890164 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.890150 6073 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl" Nov 04 02:24:33.890705 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:33.890209 6073 scope.go:122] "RemoveContainer" containerID="e91355bda496d71200feab5b9663ee71bc833ff8af36f1d69be828ba91eb969a" Nov 04 02:24:34.103944 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:34.103905 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:24:34.153653 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:34.153570 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-qfzgl"] Nov 04 02:24:34.424971 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:34.424889 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" path="/var/lib/kubelet/pods/51be9b55-dcd8-447a-83e3-303454b27fc4/volumes" Nov 04 02:24:44.497667 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:44.497636 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-24ksh_dadde5dc-97f2-479d-baa2-5f8b227aac36/manager/0.log" Nov 04 02:24:44.984593 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:44.984559 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-nwmb4_7fc15fae-0042-4cb1-b873-1f4a676b976f/proxy/0.log" Nov 04 02:24:46.698234 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:46.698206 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-019d8c20-8xq78_d44836a2-f66f-470f-9d43-1b11e3cd82cf/revoke-keys/0.log" Nov 04 02:24:48.597740 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:48.597714 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-b545d79e-mtxjd_37a57895-a14c-43dc-a70f-204eec6a274f/revoke-keys/0.log" Nov 04 02:24:48.819884 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:48.819854 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-be96d50e-bjh2b_51ed78e3-67b5-4ae4-9443-3d454a19e1f8/revoke-keys/0.log" Nov 04 02:24:49.060660 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:49.060627 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c70a9c61-wnj2l_3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba/revoke-keys/0.log" Nov 04 02:24:49.293972 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:49.293940 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d5ae9488-6fg4m_ebeca20a-2256-4382-abc3-2ca428041bb0/revoke-keys/0.log" Nov 04 02:24:49.755044 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:49.755012 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-e24ba4ae-k44tw_51db9f6c-006e-4023-aca0-abfd9b2a6f24/revoke-keys/0.log" Nov 04 02:24:52.583969 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:52.583932 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sp8n4_8404641c-60cb-4359-b0a7-f61d203d16ea/manager/0.log" Nov 04 02:24:53.860683 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:24:53.860645 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:24:54.319907 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:54.319878 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-5x5qn_a304204a-1c02-4488-a994-c073e37fa54c/manager/0.log" Nov 04 02:24:55.560185 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:55.560156 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-549f9d654fjff5d_079bd183-155c-40e7-ab79-41757399ccc5/istio-proxy/0.log" Nov 04 02:24:56.342406 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:56.342379 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56b54d794b-t6xrr_f22c973b-668b-4707-a5eb-5673763e5e6e/kube-auth-proxy/0.log" Nov 04 02:24:56.833271 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:56.833240 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-dmst2_98fc712a-69ab-4d51-aa0f-8feda896bc2c/istio-proxy/0.log" Nov 04 02:24:57.366942 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:57.366907 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j_1e1dfd17-3ead-4067-b9b3-79fe50be2195/istio-proxy/0.log" Nov 04 02:24:59.505528 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:24:59.505502 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf_41757265-bd34-464f-8420-61e0771a36c5/main/0.log" Nov 04 02:25:00.084731 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:00.084701 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-r2rjc_c2b6848b-96c0-404c-b80e-9b3dd515ac64/main/0.log" Nov 04 02:25:00.390149 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:00.390070 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-wlwzb_42d3046d-924e-4072-9f00-135939bbea7c/main/0.log" Nov 04 02:25:00.934594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:00.934563 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-8m2jn_e49e3b05-2a7b-409a-b45f-e0060cf492ea/main/0.log" Nov 04 02:25:01.486579 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:01.486551 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72_aecb1401-03fc-432e-b009-c1e05ce65482/main/0.log" Nov 04 02:25:15.027611 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:15.027521 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-24ksh_dadde5dc-97f2-479d-baa2-5f8b227aac36/manager/0.log" Nov 04 02:25:15.493299 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:15.493218 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-nwmb4_7fc15fae-0042-4cb1-b873-1f4a676b976f/proxy/0.log" Nov 04 02:25:16.434516 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:16.434489 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-019d8c20-8xq78_d44836a2-f66f-470f-9d43-1b11e3cd82cf/revoke-keys/0.log" Nov 04 02:25:18.499715 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:18.499684 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-b545d79e-mtxjd_37a57895-a14c-43dc-a70f-204eec6a274f/revoke-keys/0.log" Nov 04 02:25:18.765752 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:18.765728 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-be96d50e-bjh2b_51ed78e3-67b5-4ae4-9443-3d454a19e1f8/revoke-keys/0.log" Nov 04 02:25:19.006947 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:19.006913 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c70a9c61-wnj2l_3d3cf33e-9707-4ec2-8def-3d6a7a8c3cba/revoke-keys/0.log" Nov 04 02:25:19.247629 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:19.247547 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d5ae9488-6fg4m_ebeca20a-2256-4382-abc3-2ca428041bb0/revoke-keys/0.log" Nov 04 02:25:19.737443 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:19.737410 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-e24ba4ae-k44tw_51db9f6c-006e-4023-aca0-abfd9b2a6f24/revoke-keys/0.log" Nov 04 02:25:21.652507 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:21.652469 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sp8n4_8404641c-60cb-4359-b0a7-f61d203d16ea/manager/0.log" Nov 04 02:25:23.233915 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:23.233880 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-5x5qn_a304204a-1c02-4488-a994-c073e37fa54c/manager/0.log" Nov 04 02:25:23.858250 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:25:23.858216 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:25:23.988588 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:23.988552 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-549f9d654fjff5d_079bd183-155c-40e7-ab79-41757399ccc5/istio-proxy/0.log" Nov 04 02:25:24.770485 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:24.770460 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56b54d794b-t6xrr_f22c973b-668b-4707-a5eb-5673763e5e6e/kube-auth-proxy/0.log" Nov 04 02:25:25.277425 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:25.277396 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-dmst2_98fc712a-69ab-4d51-aa0f-8feda896bc2c/istio-proxy/0.log" Nov 04 02:25:25.817650 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:25.817621 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-95cb67c75-49d5j_1e1dfd17-3ead-4067-b9b3-79fe50be2195/istio-proxy/0.log" Nov 04 02:25:27.258358 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:27.258310 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-hxpzf_41757265-bd34-464f-8420-61e0771a36c5/main/0.log" Nov 04 02:25:27.795736 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:27.795709 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-r2rjc_c2b6848b-96c0-404c-b80e-9b3dd515ac64/main/0.log" Nov 04 02:25:28.093739 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:28.093657 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-wlwzb_42d3046d-924e-4072-9f00-135939bbea7c/main/0.log" Nov 04 02:25:28.590807 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:28.590773 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-8m2jn_e49e3b05-2a7b-409a-b45f-e0060cf492ea/main/0.log" Nov 04 02:25:29.097368 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:25:29.097342 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-26k72_aecb1401-03fc-432e-b009-c1e05ce65482/main/0.log" Nov 04 02:25:53.857815 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:25:53.857776 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:26:23.858819 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:26:23.858778 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:26:35.056133 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:35.056094 6073 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78"] Nov 04 02:26:35.105677 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:35.105641 6073 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-019d8c20-8xq78"] Nov 04 02:26:36.424992 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:36.424959 6073 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d44836a2-f66f-470f-9d43-1b11e3cd82cf" path="/var/lib/kubelet/pods/d44836a2-f66f-470f-9d43-1b11e3cd82cf/volumes" Nov 04 02:26:44.240590 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:44.240503 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tjxq6_c66eb7a2-b11d-422a-b75c-4dc43e4efe5d/global-pull-secret-syncer/0.log" Nov 04 02:26:47.119454 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:47.119423 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g5dnk_fc9507ca-eeeb-4ed3-ab31-019380293e75/konnectivity-agent/0.log" Nov 04 02:26:48.469935 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:48.469890 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-45.us-west-1.compute.internal_33ceb3d0c15f7c179f81071c0e8686a7/haproxy/0.log" Nov 04 02:26:52.185578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.185548 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sp8n4_8404641c-60cb-4359-b0a7-f61d203d16ea/manager/0.log" Nov 04 02:26:52.791356 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791289 6073 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b"] Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791822 6073 cpu_manager.go:414] "containerMap: removing container" podUID="085ce452-b573-4450-8617-ec06a0eb88e0" containerName="payload-pre-processing" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791847 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="085ce452-b573-4450-8617-ec06a0eb88e0" containerName="payload-pre-processing" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791868 6073 cpu_manager.go:414] "containerMap: removing container" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791877 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791901 6073 cpu_manager.go:414] "containerMap: removing container" podUID="38b87197-3eec-47f0-a796-e3de2cc3223c" containerName="revoke-keys" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791907 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38b87197-3eec-47f0-a796-e3de2cc3223c" containerName="revoke-keys" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791920 6073 cpu_manager.go:414] "containerMap: removing container" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791927 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791936 6073 cpu_manager.go:414] "containerMap: removing container" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791945 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" Nov 04 02:26:52.791956 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791964 6073 cpu_manager.go:414] "containerMap: removing container" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791969 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791979 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791984 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.791993 6073 cpu_manager.go:414] "containerMap: removing container" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792001 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792010 6073 cpu_manager.go:414] "containerMap: removing container" podUID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" containerName="payload-processing" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792016 6073 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" containerName="payload-processing" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792092 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="b37ebe37-80b0-408c-b2e8-451e02f0be6e" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792101 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792108 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6e94ae4-8c5a-472b-b107-48e929c3caa2" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792113 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6bf0399-7d90-40fc-85f6-7310ef9f9610" containerName="payload-processing" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792119 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="085ce452-b573-4450-8617-ec06a0eb88e0" containerName="payload-pre-processing" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792125 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="51be9b55-dcd8-447a-83e3-303454b27fc4" containerName="istio-proxy" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792131 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="38b87197-3eec-47f0-a796-e3de2cc3223c" containerName="revoke-keys" Nov 04 02:26:52.792413 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.792136 6073 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ae6d5cb-03e3-4738-929a-453217c8688a" containerName="istio-proxy" Nov 04 02:26:52.795492 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.795475 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.830640 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.830607 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxdp\" (UniqueName: \"kubernetes.io/projected/a9af604c-744b-4dca-8a61-b790a09d8f23-kube-api-access-xmxdp\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.830829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.830650 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-podres\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.830829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.830668 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-sys\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.830829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.830719 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-lib-modules\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.830829 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.830808 6073 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-proc\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.849850 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.849819 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnj78\"/\"kube-root-ca.crt\"" Nov 04 02:26:52.899163 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.899132 6073 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnj78\"/\"openshift-service-ca.crt\"" Nov 04 02:26:52.899357 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.899195 6073 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pnj78\"/\"default-dockercfg-k64b5\"" Nov 04 02:26:52.901394 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.901373 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b"] Nov 04 02:26:52.931450 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931423 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxdp\" (UniqueName: \"kubernetes.io/projected/a9af604c-744b-4dca-8a61-b790a09d8f23-kube-api-access-xmxdp\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931463 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-podres\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931487 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-sys\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931510 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-lib-modules\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931561 6073 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-proc\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931588 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-podres\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931646 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931588 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-sys\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931649 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-proc\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:52.931870 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:52.931674 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9af604c-744b-4dca-8a61-b790a09d8f23-lib-modules\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:53.000193 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.000154 6073 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxdp\" (UniqueName: \"kubernetes.io/projected/a9af604c-744b-4dca-8a61-b790a09d8f23-kube-api-access-xmxdp\") pod \"perf-node-gather-daemonset-4g82b\" (UID: \"a9af604c-744b-4dca-8a61-b790a09d8f23\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:53.108974 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.108887 6073 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:53.488895 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.488854 6073 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b"] Nov 04 02:26:53.491418 ip-10-0-138-45 kubenswrapper[6073]: W1104 02:26:53.491385 6073 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda9af604c_744b_4dca_8a61_b790a09d8f23.slice/crio-afbcc4eb860fa63966c022e2880dcd0f9021345c13b4414d734d7a5297374447 WatchSource:0}: Error finding container afbcc4eb860fa63966c022e2880dcd0f9021345c13b4414d734d7a5297374447: Status 404 returned error can't find the container with id afbcc4eb860fa63966c022e2880dcd0f9021345c13b4414d734d7a5297374447 Nov 04 02:26:53.667853 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.667814 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" event={"ID":"a9af604c-744b-4dca-8a61-b790a09d8f23","Type":"ContainerStarted","Data":"28e95fad2cfe7f19007ca0eb61d9f011e85a7bf7844f43dd9437caf2ab6bf43e"} Nov 04 02:26:53.668037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.667867 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:53.668037 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:53.667890 6073 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" event={"ID":"a9af604c-744b-4dca-8a61-b790a09d8f23","Type":"ContainerStarted","Data":"afbcc4eb860fa63966c022e2880dcd0f9021345c13b4414d734d7a5297374447"} Nov 04 02:26:53.859316 ip-10-0-138-45 kubenswrapper[6073]: E1104 02:26:53.859276 6073 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7fl4d" containerName="extractor" Nov 04 02:26:54.458261 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:54.458224 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-5x5qn_a304204a-1c02-4488-a994-c073e37fa54c/manager/0.log" Nov 04 02:26:59.684599 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:59.684567 6073 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" Nov 04 02:26:59.800170 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:26:59.800109 6073 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-4g82b" podStartSLOduration=7.800096323 podStartE2EDuration="7.800096323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:26:53.781916184 +0000 UTC m=+1191.477606384" watchObservedRunningTime="2026-11-04 02:26:59.800096323 +0000 UTC m=+1197.495786522" Nov 04 02:27:00.535252 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:00.535217 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-52mz4_d0129d90-8271-46a8-b92b-e179632baf01/kube-state-metrics/0.log" Nov 04 02:27:00.994293 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:00.994214 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-52mz4_d0129d90-8271-46a8-b92b-e179632baf01/kube-rbac-proxy-main/0.log" Nov 04 02:27:01.416225 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:01.416197 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-52mz4_d0129d90-8271-46a8-b92b-e179632baf01/kube-rbac-proxy-self/0.log" Nov 04 02:27:01.841547 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:01.841518 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-db9bb74dd-5bnpg_40a78f67-2bc4-4d66-b09a-e93063641f64/metrics-server/0.log" Nov 04 02:27:02.267991 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:02.267966 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-fw6nr_825cfc8b-20b7-481c-beaa-46a414be370a/monitoring-plugin/0.log" Nov 04 02:27:03.975578 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:03.975543 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-shpnc_9d57b05f-6e23-42fb-b3ff-25479d15b6ae/node-exporter/0.log" Nov 04 02:27:04.394742 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:04.394717 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-shpnc_9d57b05f-6e23-42fb-b3ff-25479d15b6ae/kube-rbac-proxy/0.log" Nov 04 02:27:04.896434 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:04.896409 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-shpnc_9d57b05f-6e23-42fb-b3ff-25479d15b6ae/init-textfile/0.log" Nov 04 02:27:06.582711 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:06.582675 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hp6sv_caf92aa8-a824-4089-8fa8-9005b529346f/kube-rbac-proxy-main/0.log" Nov 04 02:27:07.002594 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:07.002518 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hp6sv_caf92aa8-a824-4089-8fa8-9005b529346f/kube-rbac-proxy-self/0.log" Nov 04 02:27:07.423604 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:07.423576 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hp6sv_caf92aa8-a824-4089-8fa8-9005b529346f/openshift-state-metrics/0.log" Nov 04 02:27:07.917779 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:07.917741 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/prometheus/0.log" Nov 04 02:27:08.338292 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:08.338265 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/config-reloader/0.log" Nov 04 02:27:08.760839 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:08.760753 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/thanos-sidecar/0.log" Nov 04 02:27:09.182789 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:09.182754 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/kube-rbac-proxy-web/0.log" Nov 04 02:27:09.609402 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:09.609376 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/kube-rbac-proxy/0.log" Nov 04 02:27:10.062877 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:10.062850 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/kube-rbac-proxy-thanos/0.log" Nov 04 02:27:10.484429 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:10.484353 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5336d54e-01fd-4c53-89ff-29853596b3ad/init-config-reloader/0.log" Nov 04 02:27:12.175618 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:12.175554 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5744cbb8b4-tmdfp_7014e7c0-9d7d-4306-8e68-98437feeaf4e/telemeter-client/0.log" Nov 04 02:27:12.594168 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:12.594114 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5744cbb8b4-tmdfp_7014e7c0-9d7d-4306-8e68-98437feeaf4e/reload/0.log" Nov 04 02:27:13.013852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:13.013739 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5744cbb8b4-tmdfp_7014e7c0-9d7d-4306-8e68-98437feeaf4e/kube-rbac-proxy/0.log" Nov 04 02:27:13.440043 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:13.440015 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/thanos-query/0.log" Nov 04 02:27:13.861852 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:13.861825 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/kube-rbac-proxy-web/0.log" Nov 04 02:27:14.282691 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:14.282661 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/kube-rbac-proxy/0.log" Nov 04 02:27:14.702084 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:14.701984 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/prom-label-proxy/0.log" Nov 04 02:27:15.122584 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:15.122556 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/kube-rbac-proxy-rules/0.log" Nov 04 02:27:15.542525 ip-10-0-138-45 kubenswrapper[6073]: I1104 02:27:15.542494 6073 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764b8c9c9c-hkjbg_e2e90242-f97c-45f4-abd7-4493c8d9cc42/kube-rbac-proxy-metrics/0.log"