Nov 04 05:23:58.347780 ip-10-0-139-34 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 05:23:58.347792 ip-10-0-139-34 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 05:23:58.347801 ip-10-0-139-34 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 05:23:58.348121 ip-10-0-139-34 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 05:24:08.372326 ip-10-0-139-34 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 05:24:08.372339 ip-10-0-139-34 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ea856cde4a4840c4ad7eb928c7509b91 -- Nov 04 05:26:27.906491 ip-10-0-139-34 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:26:28.304757 ip-10-0-139-34 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:28.304757 ip-10-0-139-34 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:26:28.304757 ip-10-0-139-34 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:28.304757 ip-10-0-139-34 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:28.310506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310425 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.310506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310505 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.310506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310509 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310516 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310519 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310522 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310526 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310529 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310532 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310535 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310537 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310540 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310543 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310545 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310548 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310550 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310553 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310556 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310558 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310561 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310564 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310566 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.310629 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310569 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310572 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310574 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310577 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310579 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310582 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310585 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310588 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310590 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310593 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310596 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310599 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310601 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310604 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310607 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310610 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310613 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310615 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310618 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310621 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.311086 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310623 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310625 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310628 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310631 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310633 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310636 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310639 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310642 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310645 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310647 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310650 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310652 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310655 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310657 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310660 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310663 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310665 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310668 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310671 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.311574 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310673 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310676 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310678 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310681 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310683 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310686 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310688 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310692 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310695 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310698 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310701 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310703 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310706 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310709 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310711 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310714 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310717 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310720 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310723 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310729 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.312056 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310732 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310734 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310737 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310741 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310745 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310748 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310752 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310755 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310758 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310761 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310765 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310767 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310770 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310773 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310775 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310779 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310783 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310786 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310789 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.312556 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310792 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310796 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310799 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310802 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310804 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.310807 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311704 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311712 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311715 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311719 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311722 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311724 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311728 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311730 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311733 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311735 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311738 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311741 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311743 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311746 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.313013 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311748 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311751 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311754 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311756 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311759 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311761 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311764 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311766 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311769 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311771 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311774 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311776 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311779 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311782 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311790 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311794 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311796 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311799 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311802 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311804 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.313513 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311807 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311813 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311815 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311818 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311821 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311823 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311826 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311828 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311831 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311833 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311836 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311838 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311842 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311844 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311847 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311849 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311852 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311855 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311857 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311860 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.313978 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311862 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311865 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311867 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311870 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311873 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311876 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311879 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311887 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311890 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311892 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311895 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311898 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311900 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311903 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311905 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311908 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311912 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311916 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311920 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.314818 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311923 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311926 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311929 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311932 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311935 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311938 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311940 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311943 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311947 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311951 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311955 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311957 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311960 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311963 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311965 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311968 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311970 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311973 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311976 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.315398 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311979 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311981 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311989 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311992 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311995 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.311998 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312000 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312003 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312006 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312008 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312011 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312013 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312016 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312019 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312587 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312593 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312596 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312599 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312601 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.315856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312604 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312607 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312611 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312615 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312618 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312622 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312626 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312628 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312631 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312634 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312636 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312639 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312644 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312647 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312649 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312658 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312661 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312663 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312666 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.316444 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312668 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312671 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312674 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312678 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312681 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312684 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312688 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312691 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312694 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312697 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312699 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312702 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312705 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312707 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312710 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312712 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312715 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312718 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312721 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.316895 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312723 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312726 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312728 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312731 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312733 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312736 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312740 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312742 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312745 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312748 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312756 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312759 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312762 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312765 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312767 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312770 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312772 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312775 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312777 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.317370 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312780 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312783 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312785 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312787 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312790 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312793 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312795 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312798 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312800 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312803 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312806 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312808 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312811 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312813 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312816 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312818 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312821 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312824 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312826 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312830 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.317854 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312833 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312835 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312838 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312841 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312850 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312854 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312856 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312859 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312862 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312864 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312867 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312869 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312872 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312875 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312877 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312880 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312882 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312885 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312887 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312890 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.318335 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312892 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312895 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312898 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.312900 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313865 2579 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313880 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313889 2579 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313894 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313898 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313901 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313905 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313910 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313914 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313917 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313920 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313923 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313926 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313936 2579 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313940 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313943 2579 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313946 2579 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.313949 2579 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.314977 2579 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.314993 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:26:28.318813 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.314998 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315005 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315012 2579 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315020 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315026 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315032 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315037 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315042 2579 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315053 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315058 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315075 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315088 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315095 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315099 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315211 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315215 2579 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315232 2579 flags.go:64] FLAG: --enable-server="true" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315236 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315250 2579 flags.go:64] FLAG: --event-burst="100" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315255 2579 flags.go:64] FLAG: --event-qps="50" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315258 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315261 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315264 2579 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315270 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:26:28.319422 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315274 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315277 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315281 2579 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315284 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315287 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315290 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315293 2579 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315296 2579 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315299 2579 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315302 2579 flags.go:64] FLAG: --feature-gates="" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315306 2579 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315309 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315313 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315316 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315320 2579 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315323 2579 flags.go:64] FLAG: --help="false" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315326 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315329 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315332 2579 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315336 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315341 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315345 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315348 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:26:28.320017 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315351 2579 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315354 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315357 2579 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315360 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315363 2579 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315366 2579 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315369 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315372 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315375 2579 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315378 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315381 2579 flags.go:64] FLAG: --lock-file="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315384 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315387 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315391 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315397 2579 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315400 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315403 2579 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315406 2579 flags.go:64] FLAG: --logging-format="text" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315409 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315413 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315416 2579 flags.go:64] FLAG: --manifest-url="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315419 2579 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315424 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315427 2579 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315431 2579 flags.go:64] FLAG: --max-pods="110" Nov 04 05:26:28.320583 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315434 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315437 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315440 2579 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315446 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315449 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315452 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315456 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315468 2579 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315471 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315475 2579 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315478 2579 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315481 2579 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315483 2579 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315486 2579 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315489 2579 flags.go:64] FLAG: --port="10250" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315493 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315496 2579 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-08c26e9e28702d69f" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315500 2579 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315503 2579 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315506 2579 flags.go:64] FLAG: --register-node="true" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315509 2579 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315513 2579 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315517 2579 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315520 2579 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315522 2579 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315531 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:26:28.321182 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315534 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315537 2579 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315540 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315543 2579 flags.go:64] FLAG: --runonce="false" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315546 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315549 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315552 2579 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315555 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315558 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315563 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315566 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315570 2579 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315573 2579 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315576 2579 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315579 2579 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315582 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315585 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315588 2579 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315591 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315597 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315600 2579 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315603 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315608 2579 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315611 2579 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315613 2579 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:26:28.321846 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315616 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315619 2579 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315622 2579 flags.go:64] FLAG: --v="2" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315627 2579 flags.go:64] FLAG: --version="false" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315631 2579 flags.go:64] FLAG: --vmodule="" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315636 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.315639 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315779 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315783 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315787 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315790 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315793 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315796 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315799 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315802 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315805 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315809 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315812 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315815 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315817 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315820 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.322506 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315822 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315825 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315828 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315832 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315834 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315837 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315840 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315842 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315845 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315848 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315850 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315853 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315856 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315858 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315861 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315864 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315866 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315869 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315872 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.322998 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315874 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315879 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315883 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315886 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315889 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315892 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315895 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315897 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315901 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315904 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315907 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315909 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315912 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315915 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315918 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315920 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315923 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315927 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315929 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.323478 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315932 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315934 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315937 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315940 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315943 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315946 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315948 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315951 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315953 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315956 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315958 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315961 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315964 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315966 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315969 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315971 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315974 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315977 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315979 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315982 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.323945 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315984 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315989 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315991 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315994 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315997 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.315999 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316002 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316005 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316007 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316010 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316012 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316016 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316018 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316021 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316024 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316026 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316029 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316032 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316035 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316038 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.324426 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316040 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316043 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316046 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316048 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316051 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316054 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316056 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316059 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316062 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316064 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316067 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316070 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316072 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316078 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.316106 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:28.324907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316161 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316166 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316169 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316172 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316176 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316179 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316182 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316184 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316187 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316191 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316194 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316196 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316199 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316201 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316204 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316207 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316209 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316212 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316215 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316217 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.325290 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316238 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316242 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316246 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316250 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316253 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316255 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316258 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316261 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316263 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316266 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316271 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316275 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316278 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316280 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316283 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316286 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316288 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316291 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316294 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.325769 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316296 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316299 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316301 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316305 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316307 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316311 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316315 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316318 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316321 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316324 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316326 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316329 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316332 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316335 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316338 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316340 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316343 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316345 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316348 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316350 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.326237 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316353 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316355 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316358 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316362 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316367 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316370 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316373 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316376 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316378 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316381 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316383 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316386 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316389 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316391 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316394 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316396 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316400 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316402 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316405 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.327461 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316408 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316412 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316415 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316418 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316421 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316424 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316427 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316430 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316432 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316435 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316437 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316440 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316443 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316445 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316448 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316451 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316456 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316460 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316462 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.327928 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316465 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316468 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316470 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316473 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316476 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316479 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316482 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316484 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.316487 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.323334 2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.323349 2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.323377 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.323382 2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323451 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323456 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323459 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323462 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323465 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323468 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.328399 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323470 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323473 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323476 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323478 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323481 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323484 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323486 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323489 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323491 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323494 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323497 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323499 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323502 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323505 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323507 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323510 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323513 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323515 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323518 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.328896 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323520 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323523 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323526 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323528 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323531 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323534 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323537 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323539 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323542 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323544 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323547 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323550 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323552 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323555 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323557 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323560 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323563 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323565 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323568 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323571 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.329363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323573 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323575 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323578 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323581 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323584 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323586 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323589 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323591 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323594 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323597 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323599 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323602 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323604 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323607 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323609 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323612 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323615 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323617 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323620 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.329866 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323623 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323625 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323628 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323631 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323633 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323636 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323638 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323641 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323643 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323646 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323649 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323652 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323654 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323657 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323659 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323662 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323665 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323667 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323670 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323672 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.330392 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323675 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323677 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323681 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323685 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323688 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323690 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323693 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323696 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323698 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323701 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323704 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323706 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323709 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323714 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323718 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323721 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323723 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323726 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323729 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.330856 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323731 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323734 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323737 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.323765 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323944 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323949 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323953 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323955 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323958 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323961 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323964 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323967 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323969 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323972 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323974 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323977 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.331311 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323979 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323982 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323985 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323988 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323990 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323993 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323995 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.323998 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324001 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324004 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324007 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324009 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324012 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324014 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324017 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324019 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324022 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324024 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324027 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324030 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.331690 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324033 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324035 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324038 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324040 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324043 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324045 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324048 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324050 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324053 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324055 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324058 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324060 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324063 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324066 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324068 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324071 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324073 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324075 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324078 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324081 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.332282 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324083 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324086 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324088 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324091 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324094 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324097 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324099 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324102 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324104 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324107 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324109 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324114 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324116 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324119 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324121 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324124 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324126 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324129 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324131 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.332756 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324134 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324136 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324139 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324141 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324144 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324147 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324149 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324152 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324155 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324157 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324159 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324162 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324165 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324167 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324171 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324175 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324178 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324181 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324184 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.333303 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324187 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324189 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324192 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324195 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324198 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324202 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324205 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324208 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324210 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324213 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324215 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324230 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324233 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324237 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324241 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.333813 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324244 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.324267 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324306 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324310 2579 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324313 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324315 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324318 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324321 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324323 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324326 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324328 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324331 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324334 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324336 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324339 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324341 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324344 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:28.334172 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324347 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324350 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324352 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324355 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324358 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324362 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324364 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324367 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324369 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324372 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324374 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324377 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324379 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324382 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324384 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324387 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324389 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324392 2579 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324394 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:28.334925 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324397 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324399 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324402 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324405 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324407 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324410 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324412 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324415 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324417 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324420 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324423 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324427 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324430 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324433 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324436 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324439 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324442 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324444 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324449 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:28.335418 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324452 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324454 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324457 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324459 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324462 2579 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324464 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324467 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324469 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324472 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324476 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324480 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324482 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324486 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324488 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324491 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324494 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324496 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324499 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324501 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:28.335906 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324504 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324506 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324509 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324511 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324514 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324516 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324519 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324521 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324524 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324527 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324529 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324532 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324536 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324539 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324541 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324544 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324546 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324549 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324552 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324554 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:28.336368 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324557 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324559 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324562 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324565 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324567 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324570 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324573 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324575 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324578 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324580 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324582 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324585 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324587 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:28.324590 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.324638 2579 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.326635 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.327468 2579 server.go:1014] "Starting client certificate rotation" Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.327565 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:28.336907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.327596 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:28.337392 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.329758 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:28.337392 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.331231 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:28.338958 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.338944 2579 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:26:28.342170 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.342152 2579 log.go:25] "Validated CRI v1 image API" Nov 04 05:26:28.342987 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.342966 2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:26:28.345957 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.345939 2579 fs.go:135] Filesystem UUIDs: map[22022896-99b8-437a-8927-17a8afaf1271:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b9a61f0d-b56b-461c-bcb8-3bdae1c4c0a9:/dev/nvme0n1p4] Nov 04 05:26:28.346015 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.345958 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 05:26:28.352371 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.352257 2579 manager.go:217] Machine: {Timestamp:2026-11-04 05:26:28.350832107 +0000 UTC m=+0.334486777 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100768 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211501351ed93c3ca5de16ec5b9279 SystemUUID:ec211501-351e-d93c-3ca5-de16ec5b9279 BootID:ea856cde-4a48-40c4-ad7e-b928c7509b91 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:c8:9c:18:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:c8:9c:18:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:6f:9d:36:20:e7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:26:28.352371 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.352364 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:26:28.352493 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.352437 2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:26:28.355151 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355122 2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:26:28.355291 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355153 2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-34.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:26:28.355339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355297 2579 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:26:28.355339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355305 2579 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:26:28.355339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355331 2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:28.355418 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.355343 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:28.356381 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.356371 2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:26:28.360601 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.360590 2579 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:26:28.360721 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.360712 2579 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:26:28.361350 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361341 2579 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:26:28.361380 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361354 2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:26:28.361380 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361364 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:26:28.361380 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361373 2579 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:26:28.361462 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361381 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:26:28.361735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.361716 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:28.363635 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.363613 2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:26:28.371824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.371811 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:28.373253 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373238 2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:26:28.373301 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373275 2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:26:28.373428 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373418 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:26:28.373459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373434 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:26:28.373459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373440 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:26:28.373459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373446 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:26:28.373459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373451 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:26:28.373459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.373457 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:26:28.374034 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374026 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:26:28.374063 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374035 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:26:28.374063 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374042 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:26:28.374063 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374049 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:26:28.374063 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374059 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:26:28.374211 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374067 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:26:28.374902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374893 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:26:28.374937 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.374903 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:26:28.378579 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.378565 2579 server.go:1263] "Started kubelet" Nov 04 05:26:28.378709 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.378673 2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:26:28.378834 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.378784 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:26:28.378877 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.378869 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:26:28.379302 ip-10-0-139-34 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:26:28.381674 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.381652 2579 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:26:28.383375 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.383355 2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:26:28.385390 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.385358 2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 05:26:28.387910 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.387893 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:28.388883 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.388866 2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:26:28.388991 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.388977 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:28.389518 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.389503 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:26:28.389518 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.389505 2579 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:26:28.389661 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.389529 2579 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:26:28.389661 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.389582 2579 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:26:28.389661 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.389622 2579 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:26:28.390622 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390436 2579 factory.go:153] Registering CRI-O factory Nov 04 05:26:28.390622 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390623 2579 factory.go:223] Registration of the crio container factory successfully Nov 04 05:26:28.390751 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390700 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:26:28.390751 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390709 2579 factory.go:55] Registering systemd factory Nov 04 05:26:28.390751 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390719 2579 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:26:28.390751 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390736 2579 factory.go:103] Registering Raw factory Nov 04 05:26:28.390751 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.390749 2579 manager.go:1199] Started watching for new ooms in manager Nov 04 05:26:28.391905 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.391888 2579 manager.go:319] Starting recovery of all containers Nov 04 05:26:28.402814 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.402792 2579 manager.go:324] Recovery completed Nov 04 05:26:28.407619 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.407603 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.409833 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.409810 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.409908 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.409850 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.409908 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.409865 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.410402 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.410389 2579 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:26:28.410473 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.410400 2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:26:28.410473 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.410413 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.410473 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.410421 2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:26:28.411109 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.411095 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.411145 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.411125 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.411145 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.411138 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.412599 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.412586 2579 policy_none.go:50] "Start" Nov 04 05:26:28.412673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.412611 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:26:28.412673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.412625 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:26:28.414591 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.414578 2579 policy_none.go:44] "Start" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.451657 2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454003 2579 manager.go:341] "Starting Device Plugin manager" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.454588 2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454604 2579 server.go:87] "Starting device plugin registration server" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454796 2579 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454807 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454901 2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454980 2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.454986 2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.455617 2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.455658 2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.457443 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.458564 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.458588 2579 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.458608 2579 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:26:28.470012 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.458645 2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 05:26:28.538873 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.537214 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-34.us-west-2.compute.internal.18e4c04cf60f2bb3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-34.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-34.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:28.378536883 +0000 UTC m=+0.362191551,LastTimestamp:2026-11-04 05:26:28.378536883 +0000 UTC m=+0.362191551,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-34.us-west-2.compute.internal,}" Nov 04 05:26:28.555388 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.555341 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.556131 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.556115 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.556202 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.556147 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.556202 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.556158 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.556202 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.556180 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.558755 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.558738 2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal"] Nov 04 05:26:28.558810 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.558787 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.559465 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.559449 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.559528 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.559479 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.559528 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.559493 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.564639 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.564627 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.564802 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.564787 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.564840 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.564822 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.567604 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567583 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.567604 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567606 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.567714 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567582 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.567714 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567637 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.567714 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567649 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.567714 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.567617 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.569809 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.569793 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.569873 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.569821 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.570521 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.570506 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.570584 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.570531 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.570584 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.570542 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.590997 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.590783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.591097 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.591013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.591097 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.591039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.591097 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.590815 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-34.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 05:26:28.591097 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.590799 2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-34.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 05:26:28.601884 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.601866 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.603922 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.603901 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.621294 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.621266 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 04 05:26:28.621392 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.621345 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-34.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 04 05:26:28.621465 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.621337 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-34.us-west-2.compute.internal.18e4c04cf7ecb0dd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-34.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-34.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-34.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:28.409831645 +0000 UTC m=+0.393486318,LastTimestamp:2026-11-04 05:26:28.409831645 +0000 UTC m=+0.393486318,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-34.us-west-2.compute.internal,}" Nov 04 05:26:28.621465 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.621441 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" Nov 04 05:26:28.621592 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.621529 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 04 05:26:28.661061 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.661041 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:28.661125 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.661097 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2z9ll" Nov 04 05:26:28.689916 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.689838 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-34.us-west-2.compute.internal.18e4c04cf7ed11ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-34.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-34.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-34.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:28.409856493 +0000 UTC m=+0.393511163,LastTimestamp:2026-11-04 05:26:28.409856493 +0000 UTC m=+0.393511163,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-34.us-west-2.compute.internal,}" Nov 04 05:26:28.689991 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.689916 2579 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-139-34.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691557 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691635 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691635 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691718 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691645 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691718 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691645 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.691718 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.691646 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.890879 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.890800 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:28.891739 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.891721 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:28.891824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.891752 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:28.891824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.891764 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:28.891824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.891793 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.897176 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.897160 2579 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-34.us-west-2.compute.internal\" not found" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.905091 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.905074 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.908404 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.908380 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.948160 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:28.948140 2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:28.948259 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:28.948164 2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-34.us-west-2.compute.internal\": node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.069969 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.069945 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.170340 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.170286 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.271349 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.271314 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.327250 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.327231 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 05:26:29.327535 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.327346 2579 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6110e2483d664bf1a253f79dfb04657-a868372cdfab352f.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-139-34.us-west-2.compute.internal.18e4c04cf7ed49dd\": read tcp 10.0.139.34:47474->16.58.243.136:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-139-34.us-west-2.compute.internal.18e4c04cf7ed49dd default 4949 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-34.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-34.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-34.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:28 +0000 UTC,LastTimestamp:2026-11-04 05:26:28.556161672 +0000 UTC m=+0.539816341,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-34.us-west-2.compute.internal,}" Nov 04 05:26:29.371553 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.371522 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.472001 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.471933 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.572403 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.572368 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.665206 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.665184 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:29.667270 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.667240 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:28 +0000 UTC" deadline="2028-05-07 04:32:31.267145385 +0000 UTC" Nov 04 05:26:29.667313 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.667270 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13199h6m1.599877777s" Nov 04 05:26:29.673371 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.673356 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.685466 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.685442 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:29.686358 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.686338 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:29.686437 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.686368 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:29.686437 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:29.686380 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:29.773990 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.773941 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.874716 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.874690 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:29.975100 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:29.975063 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:30.020284 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.020250 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7b8286909c543935667f4273478b4f.slice/crio-ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea WatchSource:0}: Error finding container ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea: Status 404 returned error can't find the container with id ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea Nov 04 05:26:30.020532 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.020520 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63605d34945999c60632e953468d1ece.slice/crio-8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2 WatchSource:0}: Error finding container 8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2: Status 404 returned error can't find the container with id 8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2 Nov 04 05:26:30.025206 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.025191 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:26:30.075928 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.075904 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:26:30.171821 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.171793 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.171934 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.171920 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.189817 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.189800 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:30.237339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.237315 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:30.242824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.242808 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:30.244339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.244327 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:26:30.293608 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.293553 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:30.362956 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.362936 2579 apiserver.go:52] "Watching apiserver" Nov 04 05:26:30.389051 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.389030 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:30.450580 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.450552 2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:26:30.450839 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.450824 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-vmjtb","kube-system/konnectivity-agent-qgzd8","kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p","openshift-network-diagnostics/network-check-target-fb7vr","openshift-network-operator/iptables-alerter-crv6l","openshift-dns/node-resolver-9lljt","openshift-image-registry/node-ca-ktrgr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-cfkjk","openshift-multus/multus-additional-cni-plugins-zmrbn","openshift-multus/multus-rnjxj","kube-system/global-pull-secret-syncer-dz6r4","openshift-multus/network-metrics-daemon-swb4f"] Nov 04 05:26:30.454284 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.454269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.456148 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.456129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.456232 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.456189 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:30.459395 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.459381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.459455 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.459435 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:30.461265 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.461245 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.464702 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.464198 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:30.464702 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.464551 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:30.466925 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.466902 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.469103 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.469089 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.471162 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.471147 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.473519 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.473503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.475527 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.475513 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.477633 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.477621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.479730 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.479716 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.482040 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.482007 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"fd7b8286909c543935667f4273478b4f","Type":"ContainerStarted","Data":"ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea"} Nov 04 05:26:30.482110 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.482051 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerStarted","Data":"8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2"} Nov 04 05:26:30.490389 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.490374 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:26:30.498857 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.498836 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.498949 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.498882 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.498949 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.498882 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:26:30.501708 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.501793 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501718 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.501793 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.501893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.501893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.501893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mgrt\" (UniqueName: \"kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.501893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.501893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501876 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501896 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501981 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.501994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502082 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.502115 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502156 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502184 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4747d\" (UniqueName: \"kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvwwt\" (UniqueName: \"kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502495 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgc4h\" (UniqueName: \"kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.502596 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqkfr\" (UniqueName: \"kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502631 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502800 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.502998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2slpp\" (UniqueName: \"kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.503417 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503028 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503162 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8d2\" (UniqueName: \"kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503270 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzm88\" (UniqueName: \"kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503328 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.504084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503476 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503495 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.503994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52fs5\" (UniqueName: \"kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.504805 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.504202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.505489 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.505473 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.505543 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.505504 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:26:30.505543 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.505534 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.505635 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.505573 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:26:30.514898 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.514883 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.514964 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.514942 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:26:30.515072 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515057 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:26:30.515103 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515097 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.515153 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515139 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:26:30.515242 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515213 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.515283 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515207 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:26:30.515315 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515305 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.515352 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515325 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:26:30.515352 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515334 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:26:30.515613 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515600 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:26:30.515704 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515692 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.515835 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515824 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.515893 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.515882 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:26:30.537088 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.537072 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:26:30.537157 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.537108 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:26:30.540825 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540811 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:26:30.540907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540894 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:26:30.540947 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540895 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:26:30.540987 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540951 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:26:30.540987 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540925 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.541067 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.540999 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:26:30.541735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541500 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:30.541735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541554 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:26:30.541735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541609 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:26:30.541735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541668 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.541735 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541680 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:26:30.541978 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541767 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:30.541978 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541820 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:30.541978 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541846 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:26:30.542109 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.541977 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vpxfr" Nov 04 05:26:30.605156 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605156 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605139 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605156 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605168 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605183 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4747d\" (UniqueName: \"kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605196 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605236 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvwwt\" (UniqueName: \"kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605360 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605284 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605373 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605382 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605386 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605413 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605450 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605478 2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605534 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgc4h\" (UniqueName: \"kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.605627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqkfr\" (UniqueName: \"kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605703 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605764 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605868 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605893 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605909 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.606275 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605974 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605981 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slpp\" (UniqueName: \"kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.605998 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606011 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606020 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606033 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606042 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606066 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606082 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606120 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606125 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606133 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8d2\" (UniqueName: \"kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606162 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606206 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzm88\" (UniqueName: \"kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606256 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606295 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606438 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606084 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606466 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606471 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.607694 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606519 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606542 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606573 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606588 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606627 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606630 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606631 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.606670 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.606698 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.606727 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:26:31.106710111 +0000 UTC m=+3.090364779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:30.608207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606737 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606746 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.606753 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:31.106739089 +0000 UTC m=+3.090393756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606781 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606820 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606865 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606797 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606909 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606932 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606941 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fs5\" (UniqueName: \"kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.608902 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606880 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.606970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607019 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607072 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607231 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mgrt\" (UniqueName: \"kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607314 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607325 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.609673 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607347 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607392 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607447 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607572 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607635 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607778 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607831 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610378 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.607869 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608241 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608552 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608715 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608944 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.608992 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.609322 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.610852 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.610462 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.657363 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.657332 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:30.657363 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.657364 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:30.657510 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.657372 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:30.657510 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:30.657428 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:26:31.157413336 +0000 UTC m=+3.141068009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:30.657623 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.657549 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvwwt\" (UniqueName: \"kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.658623 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.658596 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4747d\" (UniqueName: \"kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:30.658623 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.658613 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqkfr\" (UniqueName: \"kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.658860 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.658834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2slpp\" (UniqueName: \"kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.659687 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.659663 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fs5\" (UniqueName: \"kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.659847 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.659828 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mgrt\" (UniqueName: \"kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.659915 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.659837 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8d2\" (UniqueName: \"kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.659989 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.659971 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgc4h\" (UniqueName: \"kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.660039 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.660024 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzm88\" (UniqueName: \"kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.763651 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.763629 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lljt" Nov 04 05:26:30.772448 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.772427 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6725d36_ff4f_4770_a367_6bb72b84c8a6.slice/crio-a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9 WatchSource:0}: Error finding container a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9: Status 404 returned error can't find the container with id a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9 Nov 04 05:26:30.776478 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.776463 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:26:30.780818 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.780805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:26:30.784784 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.784769 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:26:30.784964 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.784945 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f11402_cfca_4c16_bc59_c4255e791348.slice/crio-4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e WatchSource:0}: Error finding container 4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e: Status 404 returned error can't find the container with id 4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e Nov 04 05:26:30.788363 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.788346 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3c380e_88e6_4290_86f1_c554eda72ad5.slice/crio-74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b WatchSource:0}: Error finding container 74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b: Status 404 returned error can't find the container with id 74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b Nov 04 05:26:30.793622 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.793601 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298a3342_66d3_480a_9b7c_fd34a4dad76b.slice/crio-8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6 WatchSource:0}: Error finding container 8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6: Status 404 returned error can't find the container with id 8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6 Nov 04 05:26:30.799513 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.799498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:30.801634 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.801619 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:26:30.807157 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.807139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:30.808772 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.808754 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535c0ff7_d319_49dd_a9f1_39b2179b9a1b.slice/crio-2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691 WatchSource:0}: Error finding container 2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691: Status 404 returned error can't find the container with id 2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691 Nov 04 05:26:30.811107 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.811091 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d8aa45_4cab_40ab_81f8_0f60ec89009b.slice/crio-e81a4879f0a0c1f746a30da25507186f4c9370fa4ad652450cfc0f558f195e3d WatchSource:0}: Error finding container e81a4879f0a0c1f746a30da25507186f4c9370fa4ad652450cfc0f558f195e3d: Status 404 returned error can't find the container with id e81a4879f0a0c1f746a30da25507186f4c9370fa4ad652450cfc0f558f195e3d Nov 04 05:26:30.815763 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.815747 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rnjxj" Nov 04 05:26:30.816000 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.815978 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d1ed65_1711_4c55_b841_b97159f1d7af.slice/crio-1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70 WatchSource:0}: Error finding container 1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70: Status 404 returned error can't find the container with id 1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70 Nov 04 05:26:30.821910 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:30.821896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:26:30.824422 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.824403 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab73326_38e7_4426_a211_20e190306dc6.slice/crio-57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8 WatchSource:0}: Error finding container 57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8: Status 404 returned error can't find the container with id 57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8 Nov 04 05:26:30.829686 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:26:30.829667 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15462ea_de3a_472b_ab01_bc0a736b63d2.slice/crio-97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc WatchSource:0}: Error finding container 97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc: Status 404 returned error can't find the container with id 97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc Nov 04 05:26:31.111393 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.111362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:31.111560 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.111399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:31.111560 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.111514 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:31.111674 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.111572 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.111553182 +0000 UTC m=+4.095207850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:31.111674 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.111635 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:31.111674 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.111671 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.111660519 +0000 UTC m=+4.095315189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:31.212231 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.212186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:31.212386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.212335 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:31.212386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.212354 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:31.212386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.212365 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:31.212539 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:31.212422 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:26:32.212405653 +0000 UTC m=+4.196060314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:31.468274 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.468176 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70"} Nov 04 05:26:31.469363 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.469339 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qgzd8" event={"ID":"535c0ff7-d319-49dd-a9f1-39b2179b9a1b","Type":"ContainerStarted","Data":"2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691"} Nov 04 05:26:31.470318 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.470294 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ktrgr" event={"ID":"298a3342-66d3-480a-9b7c-fd34a4dad76b","Type":"ContainerStarted","Data":"8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6"} Nov 04 05:26:31.471296 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.471279 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" event={"ID":"34f11402-cfca-4c16-bc59-c4255e791348","Type":"ContainerStarted","Data":"4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e"} Nov 04 05:26:31.472388 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.472365 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" event={"ID":"f0d8aa45-4cab-40ab-81f8-0f60ec89009b","Type":"ContainerStarted","Data":"e81a4879f0a0c1f746a30da25507186f4c9370fa4ad652450cfc0f558f195e3d"} Nov 04 05:26:31.473345 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.473327 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crv6l" event={"ID":"6d3c380e-88e6-4290-86f1-c554eda72ad5","Type":"ContainerStarted","Data":"74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b"} Nov 04 05:26:31.474402 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.474377 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lljt" event={"ID":"c6725d36-ff4f-4770-a367-6bb72b84c8a6","Type":"ContainerStarted","Data":"a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9"} Nov 04 05:26:31.475476 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.475455 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerStarted","Data":"97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc"} Nov 04 05:26:31.476486 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.476461 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rnjxj" event={"ID":"5ab73326-38e7-4426-a211-20e190306dc6","Type":"ContainerStarted","Data":"57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8"} Nov 04 05:26:31.545045 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.544993 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:30 +0000 UTC" deadline="2028-06-28 03:11:15.886784064 +0000 UTC" Nov 04 05:26:31.545045 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:31.545038 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14445h44m44.341748246s" Nov 04 05:26:32.118367 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.118282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:32.118367 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.118327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:32.118576 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.118454 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:32.118576 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.118513 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.118494478 +0000 UTC m=+6.102149137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:32.118686 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.118580 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:32.118686 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.118613 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.118601581 +0000 UTC m=+6.102256242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:32.218682 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.218647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:32.218885 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.218843 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:32.218885 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.218862 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:32.218885 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.218876 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:32.219046 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.218936 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.218918514 +0000 UTC m=+6.202573184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:32.461459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.461251 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:32.461459 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.461375 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:32.461459 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.461459 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:32.461711 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.461533 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:32.461711 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.461587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:32.461711 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:32.461648 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:32.545461 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.545429 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:30 +0000 UTC" deadline="2028-07-18 18:59:30.4970354 +0000 UTC" Nov 04 05:26:32.545927 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:32.545463 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14941h32m57.951576982s" Nov 04 05:26:33.505700 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:33.505578 2579 generic.go:356] "Generic (PLEG): container finished" podID="63605d34945999c60632e953468d1ece" containerID="2138b6be6e2b976bc86e8ba4704124ba825d545a86f79751ee8c809fef0d5755" exitCode=0 Nov 04 05:26:33.505700 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:33.505624 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerDied","Data":"2138b6be6e2b976bc86e8ba4704124ba825d545a86f79751ee8c809fef0d5755"} Nov 04 05:26:34.135477 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.134872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:34.135974 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.135479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:34.135974 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.135294 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.135974 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.135737 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:26:38.135718207 +0000 UTC m=+10.119372867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.135974 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.135671 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.135974 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.135780 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:38.135768747 +0000 UTC m=+10.119423408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.236862 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.236830 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:34.237041 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.237024 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:34.237112 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.237048 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:34.237112 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.237071 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:34.237215 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.237146 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:26:38.237117873 +0000 UTC m=+10.220772547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.462338 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.462468 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.462553 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.462627 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.462684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:34.462793 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:34.462758 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:34.516897 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.516859 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerStarted","Data":"31bdcfe0fd2c5e059fe7d60afdc790782d06c32b375e5e736c9499baae46a60b"} Nov 04 05:26:34.521260 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.521205 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerStarted","Data":"7f36db159bead8098c6632b44d83bb1962fac082ceb3384c1653f9491698be39"} Nov 04 05:26:34.623730 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:34.623666 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" podStartSLOduration=4.623648977 podStartE2EDuration="4.623648977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:34.623516641 +0000 UTC m=+6.607171320" watchObservedRunningTime="2026-11-04 05:26:34.623648977 +0000 UTC m=+6.607303658" Nov 04 05:26:35.529681 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:35.528840 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="7f36db159bead8098c6632b44d83bb1962fac082ceb3384c1653f9491698be39" exitCode=0 Nov 04 05:26:35.529681 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:35.529303 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"7f36db159bead8098c6632b44d83bb1962fac082ceb3384c1653f9491698be39"} Nov 04 05:26:36.459337 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.459310 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:36.459460 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:36.459436 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:36.459536 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.459506 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:36.459599 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:36.459578 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:36.460709 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.459747 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:36.460709 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:36.459849 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:36.536377 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.536341 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"fd7b8286909c543935667f4273478b4f","Type":"ContainerStarted","Data":"ceb796cd7702a12695dfb6d995893868ceb535f79d97748571213df40203b9c8"} Nov 04 05:26:36.540824 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.540796 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qgzd8" event={"ID":"535c0ff7-d319-49dd-a9f1-39b2179b9a1b","Type":"ContainerStarted","Data":"65c0dd6bf450027e7c7c11116bd57d0495d9f3cd8bc20871e6da220ad756f91e"} Nov 04 05:26:36.750082 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.749976 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qgzd8" podStartSLOduration=4.187980864 podStartE2EDuration="8.749954798s" totalImagesPullingTime="4.561973934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:36.749183925 +0000 UTC m=+8.732838604" watchObservedRunningTime="2026-11-04 05:26:36.749954798 +0000 UTC m=+8.733609477" Nov 04 05:26:36.750270 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:36.750093 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" podStartSLOduration=6.750083344 podStartE2EDuration="6.750083344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:36.642772785 +0000 UTC m=+8.626427464" watchObservedRunningTime="2026-11-04 05:26:36.750083344 +0000 UTC m=+8.733738023" Nov 04 05:26:37.546700 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:37.546018 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ktrgr" event={"ID":"298a3342-66d3-480a-9b7c-fd34a4dad76b","Type":"ContainerStarted","Data":"464bed768f0ff85e8540093c7fbf99596d35111c4e914b984f87d03fbee4982b"} Nov 04 05:26:37.549449 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:37.549413 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" event={"ID":"34f11402-cfca-4c16-bc59-c4255e791348","Type":"ContainerStarted","Data":"8832cad0ce0d2c8091bd03775ccc0ee6590f6d15e1cf017592818f272a5bc738"} Nov 04 05:26:37.653928 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:37.653146 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ktrgr" podStartSLOduration=4.011533517 podStartE2EDuration="9.653128151s" totalImagesPullingTime="5.641594634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:37.653041165 +0000 UTC m=+9.636695844" watchObservedRunningTime="2026-11-04 05:26:37.653128151 +0000 UTC m=+9.636782829" Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.171648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.171724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.171908 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.171998 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:26:46.171974994 +0000 UTC m=+18.155629654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.172061 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:38.172158 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.172093 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:26:46.172083031 +0000 UTC m=+18.155737692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:38.272431 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.272313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:38.272589 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.272468 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:38.272589 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.272491 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:38.272589 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.272502 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:38.272589 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.272554 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:26:46.27254165 +0000 UTC m=+18.256196320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:38.460031 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.459973 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:38.460159 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.460088 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:38.460205 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.460159 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:38.460275 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.460253 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:38.461528 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:38.461505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:38.462114 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:38.461612 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:39.389760 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.389734 2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:26:39.462445 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.462349 2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:26:39.389755384Z","UUID":"6da36c69-5e1a-4f13-b640-c8adb982a0c2","Handler":null,"Name":"","Endpoint":""} Nov 04 05:26:39.464207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.464187 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:26:39.464207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.464213 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:26:39.556263 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.556143 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rnjxj" event={"ID":"5ab73326-38e7-4426-a211-20e190306dc6","Type":"ContainerStarted","Data":"f6face61c460e3db3322a2ac88a875ec8e636677d961d6f0e256710a0e5c2ffa"} Nov 04 05:26:39.560699 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.560670 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" event={"ID":"34f11402-cfca-4c16-bc59-c4255e791348","Type":"ContainerStarted","Data":"abd56e3cc198c6db10b8ef352ece8e2906f19ded6a93c28907c7fca4309b65b1"} Nov 04 05:26:39.562781 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.562741 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lljt" event={"ID":"c6725d36-ff4f-4770-a367-6bb72b84c8a6","Type":"ContainerStarted","Data":"794d052e568a1d1e60238fe23dce674904d6a65f43c913adddbffccdd169f091"} Nov 04 05:26:39.668075 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:39.668023 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-rnjxj" podStartSLOduration=3.520850098 podStartE2EDuration="11.668005497s" totalImagesPullingTime="8.147155399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:39.667276038 +0000 UTC m=+11.650930721" watchObservedRunningTime="2026-11-04 05:26:39.668005497 +0000 UTC m=+11.651660175" Nov 04 05:26:40.180010 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.179964 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:40.180470 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.180447 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:40.289552 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.289435 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-9lljt" podStartSLOduration=3.883465315 podStartE2EDuration="12.289415137s" totalImagesPullingTime="8.405949822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:39.773576751 +0000 UTC m=+11.757231428" watchObservedRunningTime="2026-11-04 05:26:40.289415137 +0000 UTC m=+12.273069819" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.459732 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:40.459868 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.459981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:40.460068 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.461714 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:40.462610 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:40.461875 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:40.568103 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.568018 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="c98cb71e899e6dc0501bbb82c0595498e8545d352027cfef9f0b504fcf670985" exitCode=0 Nov 04 05:26:40.568276 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.568115 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"c98cb71e899e6dc0501bbb82c0595498e8545d352027cfef9f0b504fcf670985"} Nov 04 05:26:40.572904 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.572877 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crv6l" event={"ID":"6d3c380e-88e6-4290-86f1-c554eda72ad5","Type":"ContainerStarted","Data":"71b5052a5835ed865c0984eb19da6be703d5aafbec144bf69ebcb99b2c0e3611"} Nov 04 05:26:40.790599 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:40.790512 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-crv6l" podStartSLOduration=4.60069117 podStartE2EDuration="12.790497886s" totalImagesPullingTime="8.189806716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:40.789989968 +0000 UTC m=+12.773644647" watchObservedRunningTime="2026-11-04 05:26:40.790497886 +0000 UTC m=+12.774152564" Nov 04 05:26:41.576905 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:41.576821 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" event={"ID":"34f11402-cfca-4c16-bc59-c4255e791348","Type":"ContainerStarted","Data":"2bed39e5c07becb9cc2e2655284da4b4f9f11787322b2ff675629d323b0a7cf2"} Nov 04 05:26:41.578353 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:41.578325 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" event={"ID":"f0d8aa45-4cab-40ab-81f8-0f60ec89009b","Type":"ContainerStarted","Data":"a21d9f91f8a6236480ea12f68b20f884347380e97817a487d583f1396c6ba262"} Nov 04 05:26:41.686920 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:41.686868 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" podStartSLOduration=3.853313098 podStartE2EDuration="13.686856373s" totalImagesPullingTime="9.833543275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:41.686723877 +0000 UTC m=+13.670378556" watchObservedRunningTime="2026-11-04 05:26:41.686856373 +0000 UTC m=+13.670511052" Nov 04 05:26:41.828404 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:41.828301 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" podStartSLOduration=3.337417094 podStartE2EDuration="13.828281071s" totalImagesPullingTime="10.490863977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:41.828141396 +0000 UTC m=+13.811796075" watchObservedRunningTime="2026-11-04 05:26:41.828281071 +0000 UTC m=+13.811935750" Nov 04 05:26:42.459090 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:42.459062 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:42.459207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:42.459114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:42.459207 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:42.459130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:42.459314 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:42.459208 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:42.459314 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:42.459263 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:42.459398 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:42.459368 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:42.581668 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:42.581641 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="e40a3f9aad2728ede8d25496b0a371ca4d0e3dde367406abda822c20be2958f2" exitCode=0 Nov 04 05:26:42.582290 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:42.581706 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"e40a3f9aad2728ede8d25496b0a371ca4d0e3dde367406abda822c20be2958f2"} Nov 04 05:26:44.459500 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:44.459248 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:44.459500 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:44.459278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:44.459960 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:44.459524 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:44.459960 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:44.459308 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:44.459960 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:44.459589 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:44.459960 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:44.459659 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:44.588853 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:44.588823 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="380a1a2c992e23a8a153a44bd05705153c8de788cde5f0999b210ef60929e247" exitCode=0 Nov 04 05:26:44.589027 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:44.588875 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"380a1a2c992e23a8a153a44bd05705153c8de788cde5f0999b210ef60929e247"} Nov 04 05:26:45.392238 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:45.392198 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:45.392409 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:45.392337 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:45.392878 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:45.392861 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:26:46.236044 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.236009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:46.236560 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.236050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:46.236560 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.236191 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:46.236560 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.236265 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.236247506 +0000 UTC m=+34.219902176 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:46.236684 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.236633 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:46.236684 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.236677 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.236664985 +0000 UTC m=+34.220319655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:46.336740 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.336704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:46.336902 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.336886 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:46.336955 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.336913 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:46.336955 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.336926 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:46.337061 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.336975 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:27:02.336962774 +0000 UTC m=+34.320617435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.459600 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.459633 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:46.459605 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.459723 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.459813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:46.460014 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:46.459867 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:47.598537 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:47.598459 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="87eb447d497b3e9c6f60cf0ac7c7ceb93d105b93f5f7aa38b80a6880cad6ab18" exitCode=0 Nov 04 05:26:47.598537 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:47.598503 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"87eb447d497b3e9c6f60cf0ac7c7ceb93d105b93f5f7aa38b80a6880cad6ab18"} Nov 04 05:26:48.459832 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:48.459803 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:48.460021 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:48.459926 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:48.460021 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:48.459927 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:48.460021 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:48.460005 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:48.460169 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:48.460061 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:48.460169 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:48.460111 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:48.603953 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:48.603916 2579 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="41d89156774b33299aaf89c63e021562bf3654e5b799ace33fff9dac0395dfcc" exitCode=0 Nov 04 05:26:48.604363 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:48.603951 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerDied","Data":"41d89156774b33299aaf89c63e021562bf3654e5b799ace33fff9dac0395dfcc"} Nov 04 05:26:49.608214 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.608050 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"3fcc8c3aab63959856a6aaf474e65ba325ce317951042c6357ffed338cdf75cd"} Nov 04 05:26:49.608563 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.608309 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"d4562b1ef9d29257bbea9f8094c41fe71e88ad9b13da3719a0951bd80e96f00e"} Nov 04 05:26:49.608563 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.608351 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"82dc318ee9fdebe2d37ac956c0539c32d764c0671ea34d3f5e35682c3b8b15cd"} Nov 04 05:26:49.608563 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.608381 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"0de67bdea49c9feb5501af6172a9f3d6e3b3c11585fd3c86e8f84b3f514f4da5"} Nov 04 05:26:49.608563 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.608421 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"d3ccebe7a8648d6a3fd99dc13d7cecbfe4b4963900c0cae7c536b5fa677d6708"} Nov 04 05:26:49.611185 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.611165 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" event={"ID":"a15462ea-de3a-472b-ab01-bc0a736b63d2","Type":"ContainerStarted","Data":"9454c8a219757f673590c304300163ad92d921c51e711d55e0255cd45b04b55d"} Nov 04 05:26:49.751532 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:49.751481 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zmrbn" podStartSLOduration=8.074606781 podStartE2EDuration="21.751468244s" totalImagesPullingTime="13.676861463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:49.751072008 +0000 UTC m=+21.734726699" watchObservedRunningTime="2026-11-04 05:26:49.751468244 +0000 UTC m=+21.735122922" Nov 04 05:26:50.459136 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:50.459103 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:50.459287 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:50.459168 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:50.459287 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:50.459173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:50.459381 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:50.459295 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:50.459381 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:50.459343 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:50.459446 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:50.459418 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:50.616433 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:50.616404 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"1303258645d8e0156c58d5a1e1ffa51af24f112b50c7fd736dc90b84f61c53d1"} Nov 04 05:26:52.458997 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:52.458960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:52.458997 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:52.458960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:52.459607 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:52.459096 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:52.459607 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:52.459096 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:52.459607 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:52.459190 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:52.459607 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:52.459314 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:52.623138 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:52.623104 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"4ed69c962700446a4d020e53fe8b4cc8e2d311efb955a1472eefaf9e260351e0"} Nov 04 05:26:54.459649 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.459620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:54.460289 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.459672 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:54.460289 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.459683 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:54.460289 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:54.459780 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:54.460289 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:54.459827 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:54.460289 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:54.459884 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:54.629738 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.629704 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" event={"ID":"d1d1ed65-1711-4c55-b841-b97159f1d7af","Type":"ContainerStarted","Data":"06674c390d1bf17f20e5ca4465ea1f14f509e10516b66d9584c6de13f9fe07a0"} Nov 04 05:26:54.630092 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.630064 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:54.630214 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.630134 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:54.630214 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.630164 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:54.645205 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.645183 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:54.645375 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.645363 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:26:54.888438 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:54.888386 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" podStartSLOduration=8.369904271 podStartE2EDuration="26.888373598s" totalImagesPullingTime="18.518469327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:54.887992385 +0000 UTC m=+26.871647076" watchObservedRunningTime="2026-11-04 05:26:54.888373598 +0000 UTC m=+26.872028276" Nov 04 05:26:56.459976 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.459719 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:56.460386 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.459731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:56.460386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.460012 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:56.460386 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.459764 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:56.460386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.460063 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:56.460386 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.460155 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:56.467660 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.467637 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dz6r4"] Nov 04 05:26:56.470593 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.470573 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb7vr"] Nov 04 05:26:56.513803 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.513780 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-swb4f"] Nov 04 05:26:56.634627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.634590 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:56.634627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.634628 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:56.634831 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.634695 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:56.634831 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:56.634742 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:56.634831 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.634805 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:56.634944 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:56.634929 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:58.462858 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:58.462831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:26:58.462858 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:58.462846 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:26:58.463379 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:58.462937 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:26:58.463379 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:58.463136 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:26:58.463379 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:58.463258 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:26:58.463379 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:26:58.463350 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:26:59.888341 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:26:59.888305 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:00.147847 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:00.147772 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:00.459426 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:00.459399 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:00.459578 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:00.459461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:00.459578 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:00.459476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:00.459578 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:00.459562 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:27:00.459695 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:00.459634 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:27:00.459774 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:00.459730 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:27:02.248167 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.248133 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:02.248167 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.248166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:02.248574 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.248310 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:27:02.248574 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.248377 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret podName:54f8b1f3-a80d-47a0-bbb8-524762fdd08e nodeName:}" failed. No retries permitted until 2026-11-04 05:27:34.248361212 +0000 UTC m=+66.232015867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret") pod "global-pull-secret-syncer-dz6r4" (UID: "54f8b1f3-a80d-47a0-bbb8-524762fdd08e") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:27:02.248574 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.248310 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:27:02.248574 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.248444 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs podName:d4720d09-b1ed-4387-bc2f-5250e4bef23c nodeName:}" failed. No retries permitted until 2026-11-04 05:27:34.248432525 +0000 UTC m=+66.232087197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs") pod "network-metrics-daemon-swb4f" (UID: "d4720d09-b1ed-4387-bc2f-5250e4bef23c") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:27:02.280525 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.280494 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:02.348425 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.348395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:02.348570 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.348531 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:27:02.348570 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.348544 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:27:02.348570 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.348552 2579 projected.go:196] Error preparing data for projected volume kube-api-access-m9fxp for pod openshift-network-diagnostics/network-check-target-fb7vr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:27:02.348683 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.348595 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp podName:c7c36da8-7108-415e-a735-26d714bf56cd nodeName:}" failed. No retries permitted until 2026-11-04 05:27:34.348583486 +0000 UTC m=+66.332238156 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9fxp" (UniqueName: "kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp") pod "network-check-target-fb7vr" (UID: "c7c36da8-7108-415e-a735-26d714bf56cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:27:02.459398 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.459364 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:02.459568 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.459377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:02.459568 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.459501 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-swb4f" podUID="d4720d09-b1ed-4387-bc2f-5250e4bef23c" Nov 04 05:27:02.459568 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.459377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:02.459725 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.459574 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb7vr" podUID="c7c36da8-7108-415e-a735-26d714bf56cd" Nov 04 05:27:02.459725 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:02.459605 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dz6r4" podUID="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" Nov 04 05:27:02.517418 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.517323 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:02.833632 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.833572 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeReady" Nov 04 05:27:02.833738 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.833681 2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 05:27:02.987464 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:02.987432 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c4xxf"] Nov 04 05:27:03.044651 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.044615 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-scktk"] Nov 04 05:27:03.044792 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.044738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.054837 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.054813 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4xxf"] Nov 04 05:27:03.054954 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.054856 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scktk"] Nov 04 05:27:03.054954 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.054940 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.094993 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.094944 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:27:03.094993 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.094970 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:27:03.095211 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.095197 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:27:03.103245 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.103214 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:27:03.103337 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.103324 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:27:03.103337 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.103332 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:27:03.103419 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.103363 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:27:03.154517 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154493 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.154617 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154522 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hq46\" (UniqueName: \"kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.154617 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154541 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.154692 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.154725 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.154767 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.154725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pscgt\" (UniqueName: \"kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.255736 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.255736 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pscgt\" (UniqueName: \"kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255812 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hq46\" (UniqueName: \"kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.255837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:03.255944 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:03.256007 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls podName:985a8caa-672c-481d-95bb-3c1784c85236 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:03.755990542 +0000 UTC m=+35.739645212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls") pod "dns-default-c4xxf" (UID: "985a8caa-672c-481d-95bb-3c1784c85236") : secret "dns-default-metrics-tls" not found Nov 04 05:27:03.256197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.256126 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.256435 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.256368 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.259668 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.259612 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.315064 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.315041 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pscgt\" (UniqueName: \"kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.315157 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.315094 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hq46\" (UniqueName: \"kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.375476 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.375429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:03.386319 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.386296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:03.593374 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.593188 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:03.759540 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.759512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:03.759695 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:03.759642 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:03.759736 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:03.759702 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls podName:985a8caa-672c-481d-95bb-3c1784c85236 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:04.759689191 +0000 UTC m=+36.743343847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls") pod "dns-default-c4xxf" (UID: "985a8caa-672c-481d-95bb-3c1784c85236") : secret "dns-default-metrics-tls" not found Nov 04 05:27:03.823057 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:03.823029 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scktk"] Nov 04 05:27:03.832383 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:03.832358 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d71858_ad31_49c4_a7a8_279e5c15e82d.slice/crio-946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd WatchSource:0}: Error finding container 946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd: Status 404 returned error can't find the container with id 946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd Nov 04 05:27:04.458909 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.458870 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:04.459369 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.458942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:04.459369 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.458939 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:04.507827 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.507800 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:04.507964 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.507869 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:04.507964 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.507892 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:04.554134 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.554109 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:04.554269 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.554150 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:04.554269 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.554162 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:04.554339 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.554290 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:04.651691 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.651655 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scktk" event={"ID":"95d71858-ad31-49c4-a7a8-279e5c15e82d","Type":"ContainerStarted","Data":"946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd"} Nov 04 05:27:04.767070 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:04.767045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:04.767211 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:04.767194 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:04.767284 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:04.767274 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls podName:985a8caa-672c-481d-95bb-3c1784c85236 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.76725721 +0000 UTC m=+38.750911882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls") pod "dns-default-c4xxf" (UID: "985a8caa-672c-481d-95bb-3c1784c85236") : secret "dns-default-metrics-tls" not found Nov 04 05:27:06.779808 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.779771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:06.780380 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:06.779950 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:06.780380 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:06.780016 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls podName:985a8caa-672c-481d-95bb-3c1784c85236 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.77999965 +0000 UTC m=+42.763654323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls") pod "dns-default-c4xxf" (UID: "985a8caa-672c-481d-95bb-3c1784c85236") : secret "dns-default-metrics-tls" not found Nov 04 05:27:06.883551 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.883519 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8hnv7"] Nov 04 05:27:06.900084 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.900063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.944305 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.944276 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8hnv7"] Nov 04 05:27:06.981886 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.981837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.981886 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.981884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh2qb\" (UniqueName: \"kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.982053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.981947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.982053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.981979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.982053 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.982003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:06.989538 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:06.989515 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:07.047911 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.047835 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:07.047911 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.047854 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:27:07.047911 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.047877 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:27:07.048073 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.047922 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:07.083212 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.083212 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2qb\" (UniqueName: \"kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.083389 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.083389 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.083301 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.083389 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.083389 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.083364 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.583347601 +0000 UTC m=+39.567002271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : configmap "kube-rbac-proxy" not found Nov 04 05:27:07.083389 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.083597 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.083415 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.083597 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.083450 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.583440125 +0000 UTC m=+39.567094795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.083597 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.083577 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.094767 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.094740 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.140790 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.140766 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2qb\" (UniqueName: \"kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.154878 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.154853 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:07.360731 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.360637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:07.588203 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.588167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.588402 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.588266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:07.588402 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.588334 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.588402 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.588391 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.588378788 +0000 UTC m=+40.572033449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.588562 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.588334 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.588562 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:07.588461 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.588449399 +0000 UTC m=+40.572104074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : configmap "kube-rbac-proxy" not found Nov 04 05:27:07.659274 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.659147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:07.865245 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:07.865189 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:08.064142 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.064109 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:08.481649 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.481619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:08.596167 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.596132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:08.596318 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.596195 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:08.596318 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:08.596273 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:08.596318 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:08.596299 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:08.596444 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:08.596334 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.596320114 +0000 UTC m=+42.579974770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : configmap "kube-rbac-proxy" not found Nov 04 05:27:08.596444 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:08.596349 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.596342964 +0000 UTC m=+42.579997620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:08.683818 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.683795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:08.734807 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.734648 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7"] Nov 04 05:27:08.749238 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.749205 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:08.792375 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.792340 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7"] Nov 04 05:27:08.802903 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.802873 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 05:27:08.837502 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.837474 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:08.837502 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.837501 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2ltvz\"" Nov 04 05:27:08.890197 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.890160 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:08.898301 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.898274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbj9\" (UniqueName: \"kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9\") pod \"migrator-66df5d496b-ckpv7\" (UID: \"1d754d22-453a-4115-9b0e-feb477099353\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:08.938436 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.938406 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-8klfn"] Nov 04 05:27:08.974585 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.974558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:08.995859 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.995796 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-8klfn"] Nov 04 05:27:08.999305 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:08.999281 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbj9\" (UniqueName: \"kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9\") pod \"migrator-66df5d496b-ckpv7\" (UID: \"1d754d22-453a-4115-9b0e-feb477099353\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:09.024551 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.024525 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Nov 04 05:27:09.024693 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.024671 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sxrvr\"" Nov 04 05:27:09.024693 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.024671 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Nov 04 05:27:09.024693 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.024692 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Nov 04 05:27:09.024849 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.024711 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Nov 04 05:27:09.055880 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.055861 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbj9\" (UniqueName: \"kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9\") pod \"migrator-66df5d496b-ckpv7\" (UID: \"1d754d22-453a-4115-9b0e-feb477099353\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:09.078488 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.078468 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:09.095508 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.095488 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:09.099938 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.099921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.100007 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.099968 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fh8g\" (UniqueName: \"kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.100144 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.100127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.200953 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.200922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.201096 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.200979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.201096 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.201088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fh8g\" (UniqueName: \"kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.201758 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.201728 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.203653 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.203634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.258274 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.258172 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fh8g\" (UniqueName: \"kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.294138 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.294111 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:09.304191 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.304174 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:09.463340 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.463311 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7"] Nov 04 05:27:09.471640 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:09.471611 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d754d22_453a_4115_9b0e_feb477099353.slice/crio-52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3 WatchSource:0}: Error finding container 52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3: Status 404 returned error can't find the container with id 52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3 Nov 04 05:27:09.663591 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.663520 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scktk" event={"ID":"95d71858-ad31-49c4-a7a8-279e5c15e82d","Type":"ContainerStarted","Data":"41c2ec73411ee52ac5036c6f955ae6feaadd2e630e748b63266e09ef22d18970"} Nov 04 05:27:09.664636 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.664615 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" event={"ID":"1d754d22-453a-4115-9b0e-feb477099353","Type":"ContainerStarted","Data":"52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3"} Nov 04 05:27:09.701759 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.701732 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-8klfn"] Nov 04 05:27:09.709876 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:09.709848 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69feb234_dfe8_438d_a0c8_f26e159f0cbf.slice/crio-24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d WatchSource:0}: Error finding container 24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d: Status 404 returned error can't find the container with id 24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d Nov 04 05:27:09.774032 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:09.773984 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-scktk" podStartSLOduration=2.990167615 podStartE2EDuration="7.773970984s" totalImagesPullingTime="4.783803369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:09.773270422 +0000 UTC m=+41.756925100" watchObservedRunningTime="2026-11-04 05:27:09.773970984 +0000 UTC m=+41.757625661" Nov 04 05:27:10.008486 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:10.008460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:10.613251 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:10.613201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:10.613406 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:10.613304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:10.613406 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.613361 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:10.613478 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.613424 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.613411193 +0000 UTC m=+46.597065849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:10.613478 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.613366 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:10.613548 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.613527 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.613509852 +0000 UTC m=+46.597164528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : configmap "kube-rbac-proxy" not found Nov 04 05:27:10.667348 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:10.667309 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" event={"ID":"69feb234-dfe8-438d-a0c8-f26e159f0cbf","Type":"ContainerStarted","Data":"24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d"} Nov 04 05:27:10.814787 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:10.814754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:10.814939 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.814924 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:10.815002 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:10.814992 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls podName:985a8caa-672c-481d-95bb-3c1784c85236 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:18.814974036 +0000 UTC m=+50.798628708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls") pod "dns-default-c4xxf" (UID: "985a8caa-672c-481d-95bb-3c1784c85236") : secret "dns-default-metrics-tls" not found Nov 04 05:27:11.732909 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:11.732879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:12.826963 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:12.826937 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:13.675761 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:13.675578 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" event={"ID":"1d754d22-453a-4115-9b0e-feb477099353","Type":"ContainerStarted","Data":"ce20ed76ac25a28644a5d83d4e7766fd9b23fd86248cf9b39b0ff84e966d69a2"} Nov 04 05:27:13.782626 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:13.782576 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" podStartSLOduration=2.435016598 podStartE2EDuration="5.782564363s" totalImagesPullingTime="3.347547765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:13.782472985 +0000 UTC m=+45.766127662" watchObservedRunningTime="2026-11-04 05:27:13.782564363 +0000 UTC m=+45.766219041" Nov 04 05:27:14.643382 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:14.643298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:14.643855 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:14.643469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:14.643855 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:14.643642 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.643855 ip-10-0-139-34 kubenswrapper[2579]: E1104 05:27:14.643732 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls podName:edc2d36d-fca3-4540-a8e8-27ea66fea7e3 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:22.643711278 +0000 UTC m=+54.627365934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8hnv7" (UID: "edc2d36d-fca3-4540-a8e8-27ea66fea7e3") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.643965 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:14.643860 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:14.679062 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:14.679037 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" event={"ID":"69feb234-dfe8-438d-a0c8-f26e159f0cbf","Type":"ContainerStarted","Data":"70d706fa74f2eba8aaa846c55f90b06ca2d26d8233dd967321f01d0db5d6db5c"} Nov 04 05:27:14.786244 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:14.786175 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" podStartSLOduration=2.245001953 podStartE2EDuration="6.786158447s" totalImagesPullingTime="4.541156494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:14.785165952 +0000 UTC m=+46.768820630" watchObservedRunningTime="2026-11-04 05:27:14.786158447 +0000 UTC m=+46.769813126" Nov 04 05:27:15.371742 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:15.371711 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:15.574371 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:15.574338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:15.787906 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:15.787875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:16.176957 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:16.176891 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9lljt_c6725d36-ff4f-4770-a367-6bb72b84c8a6/dns-node-resolver/0.log" Nov 04 05:27:16.277876 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:16.277842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:16.476607 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:16.476581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:16.679052 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:16.679009 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:16.977127 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:16.977104 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ktrgr_298a3342-66d3-480a-9b7c-fd34a4dad76b/node-ca/0.log" Nov 04 05:27:17.283490 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:17.283411 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:17.483880 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:17.483846 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:17.754572 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:17.754541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:17.962172 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:17.962140 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:17.976636 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:17.976617 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-scktk_95d71858-ad31-49c4-a7a8-279e5c15e82d/serve-healthcheck-canary/0.log" Nov 04 05:27:18.163866 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.163785 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:18.177091 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.177067 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-ckpv7_1d754d22-453a-4115-9b0e-feb477099353/migrator/0.log" Nov 04 05:27:18.363819 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.363782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:18.564943 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.564907 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:18.875981 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.875906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:18.879451 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.879431 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:18.966366 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:18.966342 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:19.333508 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:19.333479 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4xxf"] Nov 04 05:27:19.343588 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:19.343564 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985a8caa_672c_481d_95bb_3c1784c85236.slice/crio-5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1 WatchSource:0}: Error finding container 5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1: Status 404 returned error can't find the container with id 5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1 Nov 04 05:27:19.520282 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:19.520246 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:19.691030 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:19.690958 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4xxf" event={"ID":"985a8caa-672c-481d-95bb-3c1784c85236","Type":"ContainerStarted","Data":"5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1"} Nov 04 05:27:19.745742 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:19.745715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:19.946038 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:19.945965 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:20.148979 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:20.148938 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:20.354075 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:20.354040 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:20.578385 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:20.578354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:20.786505 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:20.786460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:21.750473 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:21.750442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:21.949756 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:21.949719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:22.153416 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.153393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:22.359892 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.359816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:22.565646 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.565613 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:22.697250 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.697165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:22.699496 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.699462 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4xxf" event={"ID":"985a8caa-672c-481d-95bb-3c1784c85236","Type":"ContainerStarted","Data":"b6f2814cfef9074d29047be4918ee8af43d72c6367f2a14f55b7758e4062f83d"} Nov 04 05:27:22.699629 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.699511 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4xxf" event={"ID":"985a8caa-672c-481d-95bb-3c1784c85236","Type":"ContainerStarted","Data":"980125a5bebf501974ab47d10cc4f5402cc024f61d7475b1736694709bfa586d"} Nov 04 05:27:22.699629 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.699589 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:22.699845 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.699826 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:22.809229 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.809171 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-c4xxf" podStartSLOduration=18.060252875 podStartE2EDuration="20.809158087s" totalImagesPullingTime="2.748905212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:22.808603806 +0000 UTC m=+54.792258484" watchObservedRunningTime="2026-11-04 05:27:22.809158087 +0000 UTC m=+54.792812762" Nov 04 05:27:22.855301 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:22.855273 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:23.230437 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.230409 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8hnv7"] Nov 04 05:27:23.238577 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:23.238553 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc2d36d_fca3_4540_a8e8_27ea66fea7e3.slice/crio-0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e WatchSource:0}: Error finding container 0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e: Status 404 returned error can't find the container with id 0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e Nov 04 05:27:23.359336 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.359299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:23.562777 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.562696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:23.703721 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.703690 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8hnv7" event={"ID":"edc2d36d-fca3-4540-a8e8-27ea66fea7e3","Type":"ContainerStarted","Data":"b9de583c5fb4fc9461d41f591d0ea5cb453daf2de3e48b1a77012e686b3519f9"} Nov 04 05:27:23.703869 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.703749 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8hnv7" event={"ID":"edc2d36d-fca3-4540-a8e8-27ea66fea7e3","Type":"ContainerStarted","Data":"0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e"} Nov 04 05:27:23.770341 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.770311 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:23.972618 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:23.972584 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:24.172418 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:24.172380 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:24.377333 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:24.377251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:24.576941 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:24.576909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:24.774894 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:24.774868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:25.002326 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:25.002299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:25.204249 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:25.204200 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:25.552348 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:25.552313 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:25.710665 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:25.710624 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8hnv7" event={"ID":"edc2d36d-fca3-4540-a8e8-27ea66fea7e3","Type":"ContainerStarted","Data":"5e780b0253ad57efb84b8b8569f00be9d2c11e0a1e95eeaddc686c799887a095"} Nov 04 05:27:25.954812 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:25.954739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:26.155548 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:26.155511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:26.353986 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:26.353955 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:26.646258 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:26.646172 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:26.751591 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:26.751563 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:26.962014 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:26.961936 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:27.168499 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:27.168470 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:27.368149 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:27.368115 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:27.719430 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:27.719403 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8hnv7" event={"ID":"edc2d36d-fca3-4540-a8e8-27ea66fea7e3","Type":"ContainerStarted","Data":"adb75584e3dc576b6e21095fbb5134ce9c12f996c7db40c3a0bc663ebd3160bf"} Nov 04 05:27:27.830052 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:27.830009 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8hnv7" podStartSLOduration=18.025282562 podStartE2EDuration="21.82999473s" totalImagesPullingTime="3.804712168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:27:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:27.829361497 +0000 UTC m=+59.813016174" watchObservedRunningTime="2026-11-04 05:27:27.82999473 +0000 UTC m=+59.813649407" Nov 04 05:27:32.107562 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.107532 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:27:32.111529 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.111510 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.164769 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.164741 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.164769 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.164771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.164917 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.164794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.164917 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.164815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.165003 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.164975 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.165046 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.165039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.165132 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.165112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.171073 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.171048 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:27:32.174380 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.174361 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 05:27:32.216558 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.216529 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 05:27:32.216558 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.216541 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 05:27:32.216558 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.216539 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 05:27:32.216710 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.216541 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 05:27:32.216710 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.216534 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 05:27:32.265906 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.265881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.265992 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.265928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.265992 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.265947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.265992 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.265961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266096 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266139 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266114 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266174 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266384 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266363 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266700 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266680 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266775 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266761 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.266835 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.266820 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.268334 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.268317 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.274426 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.274410 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 05:27:32.274658 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.274641 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sxw5c\"" Nov 04 05:27:32.278545 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.278526 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.300534 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.300512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:32.324907 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.324882 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.422036 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.421957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:32.508505 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.508478 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:32.706336 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.706267 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:32.793051 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.793020 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:27:32.798995 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:32.798968 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:32.827281 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:32.827254 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1148a311_091d_4817_98d0_9d6c4575acdf.slice/crio-dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac WatchSource:0}: Error finding container dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac: Status 404 returned error can't find the container with id dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac Nov 04 05:27:33.002957 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:33.002925 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:33.738366 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:33.738330 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85665dc646-2lvcs" event={"ID":"1148a311-091d-4817-98d0-9d6c4575acdf","Type":"ContainerStarted","Data":"dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac"} Nov 04 05:27:34.279433 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.279397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:34.279433 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.279435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:34.327175 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.327148 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:34.332654 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.332621 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:34.373097 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.373060 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:34.380563 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.380531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:34.383192 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.383166 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:34.428342 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.428317 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:34.478711 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.478688 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:34.485139 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.485109 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:34.561335 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.561251 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:34.564274 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.564253 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:34.594578 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.594555 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:34.597684 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.597664 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:34.608974 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.608955 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:34.612180 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.612161 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:34.942438 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.940105 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dz6r4"] Nov 04 05:27:34.988926 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:34.988891 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f8b1f3_a80d_47a0_bbb8_524762fdd08e.slice/crio-89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0 WatchSource:0}: Error finding container 89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0: Status 404 returned error can't find the container with id 89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0 Nov 04 05:27:34.992613 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.992593 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb7vr"] Nov 04 05:27:34.994755 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:34.994735 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-swb4f"] Nov 04 05:27:35.000345 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:35.000316 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c36da8_7108_415e_a735_26d714bf56cd.slice/crio-374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020 WatchSource:0}: Error finding container 374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020: Status 404 returned error can't find the container with id 374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020 Nov 04 05:27:35.003023 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:35.002950 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4720d09_b1ed_4387_bc2f_5250e4bef23c.slice/crio-e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8 WatchSource:0}: Error finding container e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8: Status 404 returned error can't find the container with id e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8 Nov 04 05:27:35.745452 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:35.745413 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dz6r4" event={"ID":"54f8b1f3-a80d-47a0-bbb8-524762fdd08e","Type":"ContainerStarted","Data":"89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0"} Nov 04 05:27:35.746601 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:35.746571 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-swb4f" event={"ID":"d4720d09-b1ed-4387-bc2f-5250e4bef23c","Type":"ContainerStarted","Data":"e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8"} Nov 04 05:27:35.747594 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:35.747568 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb7vr" event={"ID":"c7c36da8-7108-415e-a735-26d714bf56cd","Type":"ContainerStarted","Data":"374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020"} Nov 04 05:27:36.700412 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:36.700375 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:36.753745 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:36.753680 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85665dc646-2lvcs" event={"ID":"1148a311-091d-4817-98d0-9d6c4575acdf","Type":"ContainerStarted","Data":"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe"} Nov 04 05:27:36.863954 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:36.863890 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-85665dc646-2lvcs" podStartSLOduration=1.513535519 podStartE2EDuration="4.863870651s" totalImagesPullingTime="3.350335132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:36.863548024 +0000 UTC m=+68.847202702" watchObservedRunningTime="2026-11-04 05:27:36.863870651 +0000 UTC m=+68.847525330" Nov 04 05:27:36.903158 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:36.903129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:37.047249 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.046631 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-cw92n"] Nov 04 05:27:37.052501 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.051744 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.100518 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100478 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 05:27:37.100627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100568 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:27:37.100729 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100713 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t7km8\"" Nov 04 05:27:37.100806 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100791 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:27:37.100865 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100842 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 05:27:37.100937 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.100923 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:27:37.102216 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.102198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.102347 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.102328 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.102406 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.102361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdd4t\" (UniqueName: \"kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.102480 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.102461 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.105397 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.104960 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-cw92n"] Nov 04 05:27:37.111866 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.111755 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:37.203566 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.203537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.203727 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.203593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.203727 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.203620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdd4t\" (UniqueName: \"kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.203727 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.203645 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.204627 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.204359 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.206354 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.206329 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.206458 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.206436 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.261707 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.261678 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdd4t\" (UniqueName: \"kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.367031 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.366998 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:37.744575 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.744538 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-cw92n"] Nov 04 05:27:37.752036 ip-10-0-139-34 kubenswrapper[2579]: W1104 05:27:37.752005 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b57d48_f09f_4b30_ac48_63e558200561.slice/crio-6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72 WatchSource:0}: Error finding container 6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72: Status 404 returned error can't find the container with id 6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72 Nov 04 05:27:37.759186 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.759123 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" event={"ID":"70b57d48-f09f-4b30-ac48-63e558200561","Type":"ContainerStarted","Data":"6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72"} Nov 04 05:27:37.761594 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.761568 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-swb4f" event={"ID":"d4720d09-b1ed-4387-bc2f-5250e4bef23c","Type":"ContainerStarted","Data":"a7f440a9bc497b651a450e74e5b5ad45c08b5c860618328280d561185566dce8"} Nov 04 05:27:37.761701 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.761624 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-swb4f" event={"ID":"d4720d09-b1ed-4387-bc2f-5250e4bef23c","Type":"ContainerStarted","Data":"825a5f6884812e6da06c8291db82227aa5bd34ebeecbad67c75dc94a0d3454ce"} Nov 04 05:27:37.869394 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:37.869337 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-swb4f" podStartSLOduration=67.971801064 podStartE2EDuration="1m9.869319063s" totalImagesPullingTime="1.897517999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:37.868712142 +0000 UTC m=+69.852366833" watchObservedRunningTime="2026-11-04 05:27:37.869319063 +0000 UTC m=+69.852973743" Nov 04 05:27:38.013523 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:38.013448 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:38.217211 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:38.217143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:38.421186 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:38.421118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:39.771876 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:39.771796 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb7vr" event={"ID":"c7c36da8-7108-415e-a735-26d714bf56cd","Type":"ContainerStarted","Data":"c6053479f4fd042fffb3d8ec4d75615a272ea4e7f31ceb2d68ec024492448e7f"} Nov 04 05:27:39.772252 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:39.771924 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:40.643191 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.643156 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:40.777210 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.777131 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" event={"ID":"70b57d48-f09f-4b30-ac48-63e558200561","Type":"ContainerStarted","Data":"6b6abf4eb83f8d4d47e433e700371f84471a00f9fab612f01258c27be16199ff"} Nov 04 05:27:40.777210 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.777189 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" event={"ID":"70b57d48-f09f-4b30-ac48-63e558200561","Type":"ContainerStarted","Data":"2a78635da32793c073d54d6bbc336d434b4b8d9a37fa1799296273b6168e8966"} Nov 04 05:27:40.851575 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.851543 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:40.913484 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.913419 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" podStartSLOduration=1.171628255 podStartE2EDuration="3.913398254s" totalImagesPullingTime="2.741769999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:40.912376578 +0000 UTC m=+72.896031256" watchObservedRunningTime="2026-11-04 05:27:40.913398254 +0000 UTC m=+72.897052936" Nov 04 05:27:40.913664 ip-10-0-139-34 kubenswrapper[2579]: I1104 05:27:40.913544 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fb7vr" podStartSLOduration=68.429565414 podStartE2EDuration="1m12.913537582s" totalImagesPullingTime="4.483972168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:39.879995562 +0000 UTC m=+71.863650241" watchObservedRunningTime="2026-11-04 05:27:40.913537582 +0000 UTC m=+72.897192260" Nov 04 05:27:42.376836 ip-10-0-139-34 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 05:27:42.395561 ip-10-0-139-34 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 05:27:42.395721 ip-10-0-139-34 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 05:27:42.395886 ip-10-0-139-34 systemd[1]: kubelet.service: Consumed 3.477s CPU time. Nov 04 05:27:42.404903 ip-10-0-139-34 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:27:42.464346 ip-10-0-139-34 kubenswrapper[5104]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:42.464346 ip-10-0-139-34 kubenswrapper[5104]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:27:42.464346 ip-10-0-139-34 kubenswrapper[5104]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:42.464346 ip-10-0-139-34 kubenswrapper[5104]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:42.466683 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466594 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.466683 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466678 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.466683 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466684 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.466683 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466688 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466693 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466698 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466702 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466706 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466710 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466714 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466717 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466721 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466727 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466732 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466737 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466741 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466745 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466749 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466752 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466756 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466760 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466764 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466768 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.467008 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466772 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466776 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466780 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466796 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466801 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466805 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466809 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466813 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466817 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466821 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466825 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466830 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466835 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466839 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466846 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466853 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466858 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466863 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466869 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466874 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.467904 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466878 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466882 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466886 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466890 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466894 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466898 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466903 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466907 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466911 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466915 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466921 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466926 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466931 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466936 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466941 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466945 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466954 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466959 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466963 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.468801 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466967 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466971 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466975 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466979 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466984 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466989 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466993 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.466997 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467002 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467006 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467010 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467015 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467019 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467023 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467030 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467034 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467039 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467044 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467048 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.469664 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467052 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467057 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467062 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467066 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467070 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467074 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467078 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467082 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467087 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467092 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467096 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467101 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467105 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467110 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467115 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467119 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467123 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467129 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467134 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.470516 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467138 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467142 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467147 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467151 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467155 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.467161 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468422 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468433 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468437 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468442 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468450 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468455 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468459 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468464 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468468 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468473 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468478 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468482 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468486 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468490 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.471399 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468495 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468499 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468503 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468507 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468511 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468525 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468530 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468534 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468538 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468542 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468547 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468552 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468556 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468561 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468565 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468569 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468573 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468578 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468582 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.472305 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468586 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468590 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468594 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468601 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468605 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468609 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468614 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468618 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468630 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468635 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468639 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468643 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468647 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468654 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468661 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468667 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468672 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468677 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468682 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.473099 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468687 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468691 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468696 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468700 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468705 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468710 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468714 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468718 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468722 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468730 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468736 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468740 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468746 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468751 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468755 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468759 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468765 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468770 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468774 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.473717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468779 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468783 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468787 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468791 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468796 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468800 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468804 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468807 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468811 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468815 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468819 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468824 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468828 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468832 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468837 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468841 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468846 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468851 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468856 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.474189 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468860 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468864 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468869 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468873 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468877 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468882 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468886 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468890 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468895 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468899 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468903 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468909 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468914 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468918 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468922 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.468926 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469879 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469890 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469896 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469901 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.474666 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469906 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469911 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469916 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469920 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469924 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469928 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469932 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469947 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469952 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469956 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469961 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469965 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469969 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469973 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469977 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469984 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469990 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.469995 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470002 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.475286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470008 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470014 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470020 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470025 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470029 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470034 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470038 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470042 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470046 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470051 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470055 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470060 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470064 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470068 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470073 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470077 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470081 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470085 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470090 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.475744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470094 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470099 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470111 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470115 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470119 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470124 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470128 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470132 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470141 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470145 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470150 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470154 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470159 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470165 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470171 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470176 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470182 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470187 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470191 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470195 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.476202 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470200 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470204 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470208 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470212 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470233 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470238 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470242 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470247 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470251 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470256 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470261 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470265 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470269 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470274 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470278 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470283 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470288 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470293 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470297 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470301 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.476717 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470306 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470310 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470315 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470319 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470323 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470328 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470332 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470336 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470341 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470345 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470349 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470353 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470357 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470362 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470366 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470370 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470374 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470378 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470384 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470389 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.477212 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470393 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470400 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470405 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.470409 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470548 5104 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470560 5104 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470571 5104 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470578 5104 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470585 5104 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470591 5104 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470598 5104 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470605 5104 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470610 5104 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470615 5104 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470621 5104 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470627 5104 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470632 5104 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470637 5104 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470642 5104 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470647 5104 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470652 5104 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470657 5104 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470663 5104 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:27:42.477737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470667 5104 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470672 5104 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470678 5104 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470683 5104 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470690 5104 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470695 5104 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470700 5104 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470705 5104 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470710 5104 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470715 5104 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470723 5104 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470728 5104 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470733 5104 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470743 5104 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470748 5104 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470753 5104 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470757 5104 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470762 5104 flags.go:64] FLAG: --enable-server="true" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470768 5104 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470776 5104 flags.go:64] FLAG: --event-burst="100" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470781 5104 flags.go:64] FLAG: --event-qps="50" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470786 5104 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470791 5104 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470796 5104 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:27:42.478650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470802 5104 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470807 5104 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470812 5104 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470817 5104 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470822 5104 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470827 5104 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470832 5104 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470837 5104 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470842 5104 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470846 5104 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470851 5104 flags.go:64] FLAG: --feature-gates="" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470857 5104 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470862 5104 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470867 5104 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470873 5104 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470878 5104 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470883 5104 flags.go:64] FLAG: --help="false" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470888 5104 flags.go:64] FLAG: --hostname-override="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470895 5104 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470900 5104 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470905 5104 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470913 5104 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470919 5104 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:27:42.479653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470924 5104 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470929 5104 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470933 5104 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470938 5104 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470943 5104 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470949 5104 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470954 5104 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470959 5104 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470963 5104 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470968 5104 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470973 5104 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470978 5104 flags.go:64] FLAG: --lock-file="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470983 5104 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470988 5104 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.470993 5104 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471002 5104 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471007 5104 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471012 5104 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471017 5104 flags.go:64] FLAG: --logging-format="text" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471021 5104 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471026 5104 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471031 5104 flags.go:64] FLAG: --manifest-url="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471036 5104 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471044 5104 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471049 5104 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:27:42.480739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471055 5104 flags.go:64] FLAG: --max-pods="110" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471060 5104 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471067 5104 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471072 5104 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471077 5104 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471084 5104 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471088 5104 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471094 5104 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471104 5104 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471109 5104 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471114 5104 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471119 5104 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471125 5104 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471130 5104 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471135 5104 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471140 5104 flags.go:64] FLAG: --port="10250" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471144 5104 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471149 5104 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-08c26e9e28702d69f" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471154 5104 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471160 5104 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471164 5104 flags.go:64] FLAG: --register-node="true" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471169 5104 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471176 5104 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471181 5104 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471185 5104 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471190 5104 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:27:42.481842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471196 5104 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471201 5104 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471206 5104 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471211 5104 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471257 5104 flags.go:64] FLAG: --runonce="false" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471263 5104 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471269 5104 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471274 5104 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471282 5104 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471286 5104 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471292 5104 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471298 5104 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471303 5104 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471308 5104 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471313 5104 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471318 5104 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471323 5104 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471328 5104 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471333 5104 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471339 5104 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471348 5104 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471353 5104 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471357 5104 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471365 5104 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471369 5104 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:27:42.482993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471374 5104 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471379 5104 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471384 5104 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471388 5104 flags.go:64] FLAG: --v="2" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471395 5104 flags.go:64] FLAG: --version="false" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471401 5104 flags.go:64] FLAG: --vmodule="" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471407 5104 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.471412 5104 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471660 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471669 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471674 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471679 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471685 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471690 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471694 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471701 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471705 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471709 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471716 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471720 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471724 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471729 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.484081 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471733 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471738 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471742 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471747 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471752 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471757 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471761 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471765 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471770 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471774 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471778 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471783 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471787 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471792 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471796 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471800 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471805 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471809 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471813 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.484999 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471818 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471822 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471826 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471830 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471835 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471839 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471845 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471850 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471855 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471861 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471866 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471870 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471874 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471878 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471883 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471887 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471891 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471895 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471900 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471905 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.485595 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471909 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471914 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471918 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471922 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471926 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471931 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471935 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471940 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471944 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471948 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471953 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471957 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471961 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471966 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471970 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471975 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471979 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471983 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471989 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.486175 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471994 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.471999 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472004 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472011 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472016 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472021 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472025 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472030 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472035 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472039 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472043 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472048 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472052 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472057 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472061 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472065 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472070 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472074 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472078 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472082 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.486660 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472087 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472092 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472096 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472101 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472105 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472110 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472114 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472118 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472122 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472127 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472131 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472137 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472144 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472150 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.472193 5104 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:42.487154 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472278 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472287 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472291 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472298 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472302 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472308 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472312 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472316 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472320 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472325 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472329 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472334 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472339 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472343 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472348 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472352 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472357 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472361 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472366 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.487551 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472371 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472375 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472379 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472384 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472388 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472393 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472397 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472401 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472406 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472410 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472417 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472421 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472428 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472432 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472437 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472441 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472446 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472450 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472455 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472459 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.488130 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472464 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472468 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472473 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472477 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472481 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472487 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472491 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472495 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472499 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472504 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472508 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472512 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472516 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472521 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472528 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472533 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472538 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472543 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472547 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472552 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.488785 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472556 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472561 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472568 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472572 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472579 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472584 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472588 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472592 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472597 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472601 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472605 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472611 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472615 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472619 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472624 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472628 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472633 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472638 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472643 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472647 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.489303 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472651 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472656 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472660 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472667 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472673 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472678 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472682 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472686 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472691 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472696 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472700 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472704 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472709 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472713 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472719 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472724 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472729 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472734 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472738 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472743 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.489787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472747 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472752 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472758 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472762 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472766 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472770 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.472774 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.477018 5104 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.477034 5104 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.477051 5104 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.477062 5104 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477128 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477133 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477138 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477141 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477143 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477146 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477149 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477152 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477155 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.490286 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477157 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477160 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477163 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477166 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477169 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477171 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477174 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477177 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477180 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477182 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477186 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477189 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477192 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477195 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477198 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477200 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477203 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477206 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477209 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477211 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.490783 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477214 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477230 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477235 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477245 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477248 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477251 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477254 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477258 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477261 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477263 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477266 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477269 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477272 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477274 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477277 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477280 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477282 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477285 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477288 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.491341 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477300 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477304 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477306 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477309 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477312 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477315 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477318 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477321 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477325 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477329 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477332 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477335 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477337 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477340 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477343 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477347 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477350 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477359 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477363 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.491804 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477366 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477369 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477372 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477375 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477377 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477381 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477383 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477386 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477389 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477391 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477394 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477396 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477399 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477402 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477405 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477407 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477410 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477412 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477415 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477418 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.492274 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477427 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477430 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477433 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477435 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477438 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477441 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477443 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477446 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477449 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477452 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477454 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477463 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477466 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477469 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477472 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477474 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477477 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477480 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.492755 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.477483 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.477512 5104 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478028 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478418 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478424 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478435 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478440 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478444 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478448 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478452 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478457 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478461 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478466 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478470 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478476 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478480 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.493468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478484 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478488 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478497 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478502 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478506 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478510 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478515 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478520 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478525 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478530 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478560 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478565 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478570 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478574 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478578 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478588 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478592 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478597 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478601 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478605 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.493859 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478610 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478614 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478619 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478624 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478628 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478632 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478637 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478646 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478650 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478655 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478662 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478669 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478675 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478679 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478684 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478688 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478692 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478697 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478701 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.494359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478705 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478715 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478720 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478725 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478755 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478759 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478763 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478770 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478776 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478781 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478785 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478789 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478793 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478803 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478807 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478812 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478816 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478820 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478824 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.494812 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478828 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478832 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478837 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478841 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478846 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478850 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478854 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478864 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478868 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478872 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478876 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478880 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478885 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478889 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478894 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478898 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478902 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478907 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.478982 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479175 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.495275 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479187 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479193 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479198 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479204 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479209 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479213 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479235 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479240 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479244 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479249 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479254 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479258 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479262 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479267 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.479324 5104 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479384 5104 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:42.495758 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479394 5104 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479402 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479412 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479419 5104 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479424 5104 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479429 5104 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479433 5104 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479438 5104 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479443 5104 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479447 5104 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479451 5104 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479456 5104 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479460 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479465 5104 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479469 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479475 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479480 5104 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479484 5104 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479489 5104 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479493 5104 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:42.496198 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479498 5104 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479502 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479506 5104 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479511 5104 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479515 5104 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479520 5104 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479524 5104 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479529 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479533 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479537 5104 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479542 5104 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479546 5104 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479550 5104 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479554 5104 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479561 5104 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479566 5104 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479571 5104 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479576 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479580 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479584 5104 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:42.496794 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479589 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479593 5104 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479597 5104 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479602 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479606 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479611 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479615 5104 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479620 5104 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479624 5104 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479629 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479633 5104 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479637 5104 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479642 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479646 5104 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479650 5104 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479654 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479658 5104 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479662 5104 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479666 5104 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:42.497474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479670 5104 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479674 5104 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479678 5104 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479682 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479686 5104 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479693 5104 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479700 5104 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479706 5104 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479711 5104 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479715 5104 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479719 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479724 5104 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479728 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479732 5104 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479737 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479743 5104 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479747 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479751 5104 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479756 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479760 5104 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:42.498438 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479766 5104 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479773 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479778 5104 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479783 5104 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479787 5104 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479792 5104 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479796 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479801 5104 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479805 5104 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479809 5104 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479813 5104 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479818 5104 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479822 5104 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479827 5104 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479832 5104 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479836 5104 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479843 5104 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479851 5104 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479856 5104 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:42.498929 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479862 5104 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479867 5104 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479871 5104 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479875 5104 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479882 5104 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479887 5104 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:42.479891 5104 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.479989 5104 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.481576 5104 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.481652 5104 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.481914 5104 server.go:1014] "Starting client certificate rotation" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.482045 5104 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.482116 5104 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:28 +0000 UTC" deadline="2028-04-23 22:08:28.843253844 +0000 UTC" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.482142 5104 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12880h40m46.361113169s" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.482673 5104 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.484422 5104 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:42.499446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.485687 5104 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:27:42.499840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.487428 5104 log.go:25] "Validated CRI v1 image API" Nov 04 05:27:42.499840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.488183 5104 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:27:42.499840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.489963 5104 fs.go:135] Filesystem UUIDs: map[22022896-99b8-437a-8927-17a8afaf1271:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b9a61f0d-b56b-461c-bcb8-3bdae1c4c0a9:/dev/nvme0n1p4] Nov 04 05:27:42.499931 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.489978 5104 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e/userdata/shm major:0 minor:253 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72/userdata/shm major:0 minor:315 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0/userdata/shm major:0 minor:283 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac/userdata/shm major:0 minor:276 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8/userdata/shm major:0 minor:285 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~projected/kube-api-access-gqct5:{mountpoint:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~projected/kube-api-access-gqct5 major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:274 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1d754d22-453a-4115-9b0e-feb477099353/volumes/kubernetes.io~projected/kube-api-access-zmbj9:{mountpoint:/var/lib/kubelet/pods/1d754d22-453a-4115-9b0e-feb477099353/volumes/kubernetes.io~projected/kube-api-access-zmbj9 major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/298a3342-66d3-480a-9b7c-fd34a4dad76b/volumes/kubernetes.io~projected/kube-api-access-nzm88:{mountpoint:/var/lib/kubelet/pods/298a3342-66d3-480a-9b7c-fd34a4dad76b/volumes/kubernetes.io~projected/kube-api-access-nzm88 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34f11402-cfca-4c16-bc59-c4255e791348/volumes/kubernetes.io~projected/kube-api-access-52fs5:{mountpoint:/var/lib/kubelet/pods/34f11402-cfca-4c16-bc59-c4255e791348/volumes/kubernetes.io~projected/kube-api-access-52fs5 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/535c0ff7-d319-49dd-a9f1-39b2179b9a1b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/535c0ff7-d319-49dd-a9f1-39b2179b9a1b/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54f8b1f3-a80d-47a0-bbb8-524762fdd08e/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/54f8b1f3-a80d-47a0-bbb8-524762fdd08e/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5ab73326-38e7-4426-a211-20e190306dc6/volumes/kubernetes.io~projected/kube-api-access-8mgrt:{mountpoint:/var/lib/kubelet/pods/5ab73326-38e7-4426-a211-20e190306dc6/volumes/kubernetes.io~projected/kube-api-access-8mgrt major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~projected/kube-api-access-7fh8g:{mountpoint:/var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~projected/kube-api-access-7fh8g major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~secret/signing-key major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3c380e-88e6-4290-86f1-c554eda72ad5/volumes/kubernetes.io~projected/kube-api-access-2slpp:{mountpoint:/var/lib/kubelet/pods/6d3c380e-88e6-4290-86f1-c554eda72ad5/volumes/kubernetes.io~projected/kube-api-access-2slpp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~projected/kube-api-access-sdd4t:{mountpoint:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~projected/kube-api-access-sdd4t major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~projected/kube-api-access-6hq46:{mountpoint:/var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~projected/kube-api-access-6hq46 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~projected/kube-api-access-pscgt:{mountpoint:/var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~projected/kube-api-access-pscgt major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~secret/metrics-tls major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a15462ea-de3a-472b-ab01-bc0a736b63d2/volumes/kubernetes.io~projected/kube-api-access-kgc4h:{mountpoint:/var/lib/kubelet/pods/a15462ea-de3a-472b-ab01-bc0a736b63d2/volumes/kubernetes.io~projected/kube-api-access-kgc4h major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6725d36-ff4f-4770-a367-6bb72b84c8a6/volumes/kubernetes.io~projected/kube-api-access-hm8d2:{mountpoint:/var/lib/kubelet/pods/c6725d36-ff4f-4770-a367-6bb72b84c8a6/volumes/kubernetes.io~projected/kube-api-access-hm8d2 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7c36da8-7108-415e-a735-26d714bf56cd/volumes/kubernetes.io~projected/kube-api-access-m9fxp:{mountpoint:/var/lib/kubelet/pods/c7c36da8-7108-415e-a735-26d714bf56cd/volumes/kubernetes.io~projected/kube-api-access-m9fxp major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~projected/kube-api-access-lvwwt:{mountpoint:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~projected/kube-api-access-lvwwt major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~projected/kube-api-access-4747d:{mountpoint:/var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~projected/kube-api-access-4747d major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~secret/metrics-certs major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~projected/kube-api-access-qh2qb:{mountpoint:/var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~projected/kube-api-access-qh2qb major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:249 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~projected/kube-api-access-gqkfr:{mountpoint:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~projected/kube-api-access-gqkfr major:0 minor:61 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/047bd468ea52bbd06498f23ae016df23da77e9d42f5642e105152c5697ef68d8/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/43e46bc9b6a84162d8a77ebda0bcbc324fba0f5c8a2a0cff023812ae7d476682/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c62bea50b43bc2a0ef29e151c83e13c0f0b15b1c06354c9fb2c599c8d49e1a6d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/380d004e64790be3bbee8a30b24fac328e071c38d60652116f023080e669c90d/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/05e75536eaa054323832ef26ad873a9ede0cff02357d6123dc992efa3791afc0/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/c7117154e6a8a72d89c7add3a8136c96911b4ff9e2da50b693a1529425f29c4c/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/8a3a51288fe9026f996b8b0237fd9f413794033dfa5715c7e4b6f0ba75cbe675/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/1f9d00dfec6fb94be49b478cc91b63674f3391d68a4d3dc5c189a2165467c9e7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/029ba3c222a160dc3e090c981cd48dd5ce521c3adc1f1d89d42188818fdac958/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/d1bac4ccc381c260cc3f5f12ad1d4bb99cd5b5760a462e18f7c48fe6c259eaf7/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/5c2dfc3171443e4893d48499ce665d5d4c70e40f91224b2eba898e1f66f566a4/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/9e198610224c1ede8b003b8d3f6d08ce7eb7ca44bfab29b9920dc708be1ccdf8/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/a536375ab7aedc851a63a7ee547fc0d10784f1cf3dd098aa9f759e416edd7cff/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/05883ae20af2996d0ec08170ad4d0a476cf36bb252431ca3287f70f2d9b01f29/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/9ca73eaad1a73af0cdd720d841c4812da2d5c245a090bf2d8ed07a040503adb5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/7bbdae7eaccbb4a8edb64252338a8e30ed85c2d7ca5902fc4690a8edbac82d90/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/a1cb5d54a6ae283fcda507d4399fbfdc1467dfcb20b7954cd17400cdfb1ac18d/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/c34081e13f232a1b3def6160353319a9c5c98ee56e1964a76d95e9f41998472a/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8a9c834043538a108d4165d229b2972a14b721db707fa520f282eff49ad3ec9b/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/67528206ac846a51bbe221f99bdfd568dfdf5383f1c4726c5cee96a7467db1d9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/82b87d0c15a474fdf6aabbe401152fa0d0c33675ec93a9a43a481499f78fb950/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/2f26885a9dc318f946b0764318d53a1fb0d65e5d2a74784f4ffe73fbd7ddae14/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-207:{mountpoint:/var/lib/containers/storage/overlay/3ef2f2c3e8bf4d735e87a2abc1404abfda881737b4540ddc5ac5f2dd5d4989d0/merged major:0 minor:207 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/aa04c3cbf7bf5ff5cda73f82ae1d51a4ec7767cdd6cb9ab3f9c1144208c7e52a/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/b9d35d537e9e9a3c8387729ed390a35eccbd8c664ae16f3b2c0ac603fd113368/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/23961a2600cbca3d868603504480e08d4e1a195a56e9887c95f52c8557a68bc6/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/17bd2812e8ee0ca9ef5e7e602e316d26d497eafb39f341e45987790aeacbf20c/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/fc861e7d123988e3a907d441325981e69c4169f3e9c9842a1736a550b0d76121/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/8c0a8c11abda7b7d38b99950678493331907ea34034aaa8f23bd18940ab3e3f0/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/96ce30d4d076c1f119f10426aec18086f4c3a4fd2514e43e2e755c0932a57cf6/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/b680bb10af778db8f833e31729c34bdb9ad62876e65af6f291b07956c5f78388/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/cef9fd8559f7832592a23c56b14ee17a82b88e161d7ae9eda27cd5d3dde3f8b7/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/218cce3734cf2532077a5fc2cf20eac8fac071029da26d76dc5508ba7a676821/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/18ffdbc1bd018f8175fb6cb0168312323b10a39b38138d3826e40eec474d110e/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/c8991d2675ffc7ca388b837d63c9a4932d3b5bf3ea7084af55ebeb5a5e4ebaf7/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/8f1a156142c3c8f0255a92fab504f67aed2b5e18da63b90ad05599463c1283dc/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/6a88708e78150ea8c30929d3c647ef7fc8d2f7301b796222f234f4ab2dd852ed/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/66fc946c37ee7d2045724a016be60209c79b61249b7607214368a0181655ccc2/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/8e90e7ed6618e9a4a09956067ffab7212c46cd99d9e1b2c0d7d21a30352a5749/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-297:{mountpoint:/var/lib/containers/storage/overlay/20fab86862d7048b3b848d65c3a937fbea2e1e3f79723c5a6d26d8ab0d7afbb0/merged major:0 minor:297 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/d98014ed8ff15a1826fff84816d7df4b5f3a8edcc7418e2a425f40648868012e/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-317:{mountpoint:/var/lib/containers/storage/overlay/16ca6a31a74251b135c4637b002068e18dded00991efc4f76a586be77597435a/merged major:0 minor:317 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/70da71614e0fff9a8bfd2b1d42caebef24e3b83d1f0ef943227e1fe9341e0e6e/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-321:{mountpoint:/var/lib/containers/storage/overlay/d0b800f1290a1a4ff7fa7866b48496c5b8e04d5ca4d18a8298da16f35a358fb7/merged major:0 minor:321 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/fa5844252b7a83b7bbe7194c7d301f690af35d18b4a79d6ff431df49146c5783/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/0728b509c24cb39f84034760b99f0dcb78afdbc82348ac6dcacfa8881f8f272d/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/1bf219f3b01124ae2bc61a07f8ac59ba015fdd7d3140cf08bfa9032f31c752e4/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/6c473d840d8ad4d057c772828fc444ba736f7d28e90fe65d2ef0f3062c8ec42e/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/15e24bebf37a6b9e04bed17ff224a9f6e0bd84b79c33b40e380d9f69ce048fd7/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/0f38b6bdf9ef36d6e37248676eb3d83a982a456da8d3a382454e3b84f21f7598/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/bd9968449e82da6d52d25d9048be5374be36491a0bc559fa4de84236053faab7/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/3f9c64a5378bfa3d9b107a0af52ebc3965476686d38e4dc512159468a6cea9ae/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/a5e2fc7d545a5642f7e168e0ce8b2541e91c720d0ec25deecf5a06accbe39251/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/b0da0777efcfdf94c201058909982322580bb71e50768edd0ea4316f261fac05/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/1101bd59c263f92f2e8431b9417d91e808e674e356e405d452e38f5cc227367e/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/6b944732a68b6e157ee7f7992182c555fce80d3855fea11986ca1ae7787c65df/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.498387 5104 manager.go:217] Machine: {Timestamp:2026-11-04 05:27:42.497516546 +0000 UTC m=+0.061899837 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100255 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211501351ed93c3ca5de16ec5b9279 SystemUUID:ec211501-351e-d93c-3ca5-de16ec5b9279 BootID:ea856cde-4a48-40c4-ad7e-b928c7509b91 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:280 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:313 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~projected/kube-api-access-sdd4t DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5ab73326-38e7-4426-a211-20e190306dc6/volumes/kubernetes.io~projected/kube-api-access-8mgrt DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2cc45660b7911b22c91a45058aa22108c02778621a55b3c6a7e018e655b3ca8/userdata/shm DeviceMajor:0 DeviceMinor:285 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~projected/kube-api-access-qh2qb DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~projected/kube-api-access-6hq46 DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-317 DeviceMajor:0 DeviceMinor:317 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4720d09-b1ed-4387-bc2f-5250e4bef23c/volumes/kubernetes.io~projected/kube-api-access-4747d DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/c6725d36-ff4f-4770-a367-6bb72b84c8a6/volumes/kubernetes.io~projected/kube-api-access-hm8d2 DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/57315260da38006b18268c07c686040fd6b382d942b200197b83006b540d0bd8/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/34f11402-cfca-4c16-bc59-c4255e791348/volumes/kubernetes.io~projected/kube-api-access-52fs5 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0d745f9a1a45173a7d77b7a7f8458df7953daa7315c61db0959efbcd3157085e/userdata/shm DeviceMajor:0 DeviceMinor:253 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac/userdata/shm DeviceMajor:0 DeviceMinor:276 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8badfe5bbc3be12d43671ba7a00e005e0dee0c52986a0f1ff036c2dcae3917c6/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:274 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c7c36da8-7108-415e-a735-26d714bf56cd/volumes/kubernetes.io~projected/kube-api-access-m9fxp DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/69feb234-dfe8-438d-a0c8-f26e159f0cbf/volumes/kubernetes.io~projected/kube-api-access-7fh8g DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24e11b6c66528eb96868fc7582e0453cbc9bc725ce97ab78a325073a9dfdbf8d/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:270 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d490f6ca9779276d1adddde9382b16994a876847980feba1684f0e3a0d25691/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89c2f10a9e518c2bbcc836d6e27a9e893e1473d9d0f20ecf73d9eafa93aee7c0/userdata/shm DeviceMajor:0 DeviceMinor:283 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-297 DeviceMajor:0 DeviceMinor:297 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/a15462ea-de3a-472b-ab01-bc0a736b63d2/volumes/kubernetes.io~projected/kube-api-access-kgc4h DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52a34fb2a4204c2834507e2712b5f3c7ffb299f785c5f1994d97c231cabba7b3/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~projected/kube-api-access-lvwwt DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-207 DeviceMajor:0 DeviceMinor:207 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f0d8aa45-4cab-40ab-81f8-0f60ec89009b/volumes/kubernetes.io~projected/kube-api-access-gqkfr DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/298a3342-66d3-480a-9b7c-fd34a4dad76b/volumes/kubernetes.io~projected/kube-api-access-nzm88 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6437796291fb4dcdbffa1b51682ff9aca0a70f7cd0a2a0e91d1d8c480dbfcf72/userdata/shm DeviceMajor:0 DeviceMinor:315 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/54f8b1f3-a80d-47a0-bbb8-524762fdd08e/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:281 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/374d841b69c368a24d05c0c78db7c3f838b02266419f6e3458eab3ad7c659020/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-321 DeviceMajor:0 DeviceMinor:321 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1edb73bbae55ceaa082d434a1e35c7242605bf500857abda2cec9ae16aff7e70/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ec0e77528123d478f21ae286042a368e8696a8a7b392c7540fa6c71ba2bd7d1/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a122431ae91e9f6c0eb5136af3e6644618bf31eccee799bdee7ab433fa2fb8f9/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/535c0ff7-d319-49dd-a9f1-39b2179b9a1b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4fa30a92da2ebfbb22684ae7e256665edc4e3a792ca900d50adb7fb362ffc93e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/95d71858-ad31-49c4-a7a8-279e5c15e82d/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70b57d48-f09f-4b30-ac48-63e558200561/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:309 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1d1ed65-1711-4c55-b841-b97159f1d7af/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3c380e-88e6-4290-86f1-c554eda72ad5/volumes/kubernetes.io~projected/kube-api-access-2slpp DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/97bc09e9d2186bb76f4203864b922a3a9b424121e617eab17d8969ba3e2c95dc/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/985a8caa-672c-481d-95bb-3c1784c85236/volumes/kubernetes.io~projected/kube-api-access-pscgt DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/946364a5c46e2ecd9da776b1b216adb0ac50b3e92278de0d970ae9fc19dc23bd/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1d754d22-453a-4115-9b0e-feb477099353/volumes/kubernetes.io~projected/kube-api-access-zmbj9 DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/edc2d36d-fca3-4540-a8e8-27ea66fea7e3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:249 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74db12981c6efb2a3d6cc979d8dc9023f097b2d1b32a65bbe5a0667f66ab7a6b/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes/kubernetes.io~projected/kube-api-access-gqct5 DeviceMajor:0 DeviceMinor:275 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0d745f9a1a45173 MacAddress:16:ef:c0:02:ae:9d Speed:10000 Mtu:8901} {Name:24e11b6c66528eb MacAddress:42:1e:84:ea:68:70 Speed:10000 Mtu:8901} {Name:374d841b69c368a MacAddress:a6:8e:44:14:ed:6b Speed:10000 Mtu:8901} {Name:52a34fb2a4204c2 MacAddress:ee:a5:21:15:a8:46 Speed:10000 Mtu:8901} {Name:5ec0e77528123d4 MacAddress:6e:e9:ab:2d:e8:00 Speed:10000 Mtu:8901} {Name:6437796291fb4dc MacAddress:ce:01:75:7d:39:64 Speed:10000 Mtu:8901} {Name:89c2f10a9e518c2 MacAddress:42:ec:4d:81:7a:98 Speed:10000 Mtu:8901} {Name:946364a5c46e2ec MacAddress:b6:17:c7:02:65:34 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:c8:9c:18:75 Speed:0 Mtu:9001} {Name:br-int MacAddress:de:8a:0f:82:f4:1d Speed:0 Mtu:8901} {Name:dd807e34f40dd25 MacAddress:2e:2a:98:14:89:d8 Speed:10000 Mtu:8901} {Name:e2cc45660b7911b MacAddress:3a:72:27:2b:45:f4 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:c8:9c:18:75 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:3e:82:d9:81:fe:4a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:1a:6f:9d:36:20:e7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.498925 5104 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.498978 5104 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499272 5104 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499299 5104 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-34.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499416 5104 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499423 5104 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499433 5104 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499451 5104 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499523 5104 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499679 5104 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499787 5104 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499823 5104 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499838 5104 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499847 5104 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499856 5104 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:27:42.500129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.499864 5104 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:27:42.500719 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.500645 5104 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:42.500719 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.500659 5104 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:42.501113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.501096 5104 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:27:42.503674 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.503656 5104 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:42.503674 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.503672 5104 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:42.503884 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.503803 5104 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 05:27:42.504055 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504039 5104 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:27:42.504120 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504081 5104 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:27:42.504204 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504193 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:27:42.504274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504213 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:27:42.504274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504238 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:27:42.504274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504248 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:27:42.504274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504257 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:27:42.504274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504267 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504277 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504287 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504299 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504307 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504321 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504336 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504371 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:27:42.504512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.504381 5104 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:27:42.507766 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.507749 5104 server.go:1263] "Started kubelet" Nov 04 05:27:42.507853 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.507821 5104 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:27:42.508246 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.507949 5104 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:27:42.508336 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.508264 5104 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:27:42.508311 ip-10-0-139-34 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:27:42.508603 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.508575 5104 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:27:42.509256 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.509104 5104 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514006 5104 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514018 5104 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514096 5104 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514110 5104 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.514116 5104 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514098 5104 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:30 +0000 UTC" deadline="2028-06-06 02:46:24.708671158 +0000 UTC" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514142 5104 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13917h18m42.194532402s" Nov 04 05:27:42.514457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.514154 5104 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:27:42.515901 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515872 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d8aa45-4cab-40ab-81f8-0f60ec89009b" volumeName="kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515902 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d8aa45-4cab-40ab-81f8-0f60ec89009b" volumeName="kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515917 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="535c0ff7-d319-49dd-a9f1-39b2179b9a1b" volumeName="kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515928 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515940 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34f11402-cfca-4c16-bc59-c4255e791348" volumeName="kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515951 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="535c0ff7-d319-49dd-a9f1-39b2179b9a1b" volumeName="kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515962 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5ab73326-38e7-4426-a211-20e190306dc6" volumeName="kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config" seLinuxMountContext="" Nov 04 05:27:42.515984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515974 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b57d48-f09f-4b30-ac48-63e558200561" volumeName="kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.515989 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95d71858-ad31-49c4-a7a8-279e5c15e82d" volumeName="kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516001 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a15462ea-de3a-472b-ab01-bc0a736b63d2" volumeName="kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516013 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516025 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54f8b1f3-a80d-47a0-bbb8-524762fdd08e" volumeName="kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516037 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a15462ea-de3a-472b-ab01-bc0a736b63d2" volumeName="kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516060 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a15462ea-de3a-472b-ab01-bc0a736b63d2" volumeName="kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516078 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6725d36-ff4f-4770-a367-6bb72b84c8a6" volumeName="kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516089 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6725d36-ff4f-4770-a367-6bb72b84c8a6" volumeName="kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516101 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1d1ed65-1711-4c55-b841-b97159f1d7af" volumeName="kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516111 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1d1ed65-1711-4c55-b841-b97159f1d7af" volumeName="kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516122 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516134 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3c380e-88e6-4290-86f1-c554eda72ad5" volumeName="kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516145 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b57d48-f09f-4b30-ac48-63e558200561" volumeName="kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516157 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95d71858-ad31-49c4-a7a8-279e5c15e82d" volumeName="kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516169 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985a8caa-672c-481d-95bb-3c1784c85236" volumeName="kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516181 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985a8caa-672c-481d-95bb-3c1784c85236" volumeName="kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516193 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1d1ed65-1711-4c55-b841-b97159f1d7af" volumeName="kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516204 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4720d09-b1ed-4387-bc2f-5250e4bef23c" volumeName="kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516217 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516245 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5ab73326-38e7-4426-a211-20e190306dc6" volumeName="kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516262 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69feb234-dfe8-438d-a0c8-f26e159f0cbf" volumeName="kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516275 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b57d48-f09f-4b30-ac48-63e558200561" volumeName="kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516287 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7c36da8-7108-415e-a735-26d714bf56cd" volumeName="kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516299 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1d1ed65-1711-4c55-b841-b97159f1d7af" volumeName="kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516312 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4720d09-b1ed-4387-bc2f-5250e4bef23c" volumeName="kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516323 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edc2d36d-fca3-4540-a8e8-27ea66fea7e3" volumeName="kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516336 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca" seLinuxMountContext="" Nov 04 05:27:42.516337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516347 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d754d22-453a-4115-9b0e-feb477099353" volumeName="kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516358 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298a3342-66d3-480a-9b7c-fd34a4dad76b" volumeName="kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516370 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69feb234-dfe8-438d-a0c8-f26e159f0cbf" volumeName="kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516383 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985a8caa-672c-481d-95bb-3c1784c85236" volumeName="kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516394 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985a8caa-672c-481d-95bb-3c1784c85236" volumeName="kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516405 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1d1ed65-1711-4c55-b841-b97159f1d7af" volumeName="kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516416 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edc2d36d-fca3-4540-a8e8-27ea66fea7e3" volumeName="kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516426 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5ab73326-38e7-4426-a211-20e190306dc6" volumeName="kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516449 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70b57d48-f09f-4b30-ac48-63e558200561" volumeName="kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516461 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a15462ea-de3a-472b-ab01-bc0a736b63d2" volumeName="kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516473 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edc2d36d-fca3-4540-a8e8-27ea66fea7e3" volumeName="kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516486 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d8aa45-4cab-40ab-81f8-0f60ec89009b" volumeName="kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516498 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298a3342-66d3-480a-9b7c-fd34a4dad76b" volumeName="kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516511 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69feb234-dfe8-438d-a0c8-f26e159f0cbf" volumeName="kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516524 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516536 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1148a311-091d-4817-98d0-9d6c4575acdf" volumeName="kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516547 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3c380e-88e6-4290-86f1-c554eda72ad5" volumeName="kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516559 5104 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edc2d36d-fca3-4540-a8e8-27ea66fea7e3" volumeName="kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume" seLinuxMountContext="" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516570 5104 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.516578 5104 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.517091 5104 factory.go:55] Registering systemd factory Nov 04 05:27:42.517708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.517116 5104 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.517891 5104 factory.go:153] Registering CRI-O factory Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.517941 5104 factory.go:223] Registration of the crio container factory successfully Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.518024 5104 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.518046 5104 factory.go:103] Registering Raw factory Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.518062 5104 manager.go:1199] Started watching for new ooms in manager Nov 04 05:27:42.518493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.518465 5104 manager.go:319] Starting recovery of all containers Nov 04 05:27:42.540730 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.540688 5104 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:27:42.542286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.542261 5104 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:27:42.542286 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.542283 5104 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:27:42.542461 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.542306 5104 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:27:42.542461 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.542347 5104 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 05:27:42.558705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558658 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="41d89156774b33299aaf89c63e021562bf3654e5b799ace33fff9dac0395dfcc" exitCode=0 Nov 04 05:27:42.558705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558703 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="87eb447d497b3e9c6f60cf0ac7c7ceb93d105b93f5f7aa38b80a6880cad6ab18" exitCode=0 Nov 04 05:27:42.558847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558714 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="380a1a2c992e23a8a153a44bd05705153c8de788cde5f0999b210ef60929e247" exitCode=0 Nov 04 05:27:42.558847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558724 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="e40a3f9aad2728ede8d25496b0a371ca4d0e3dde367406abda822c20be2958f2" exitCode=0 Nov 04 05:27:42.558847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558733 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="c98cb71e899e6dc0501bbb82c0595498e8545d352027cfef9f0b504fcf670985" exitCode=0 Nov 04 05:27:42.558847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.558743 5104 generic.go:356] "Generic (PLEG): container finished" podID="a15462ea-de3a-472b-ab01-bc0a736b63d2" containerID="7f36db159bead8098c6632b44d83bb1962fac082ceb3384c1653f9491698be39" exitCode=0 Nov 04 05:27:42.581991 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.581960 5104 generic.go:356] "Generic (PLEG): container finished" podID="63605d34945999c60632e953468d1ece" containerID="2138b6be6e2b976bc86e8ba4704124ba825d545a86f79751ee8c809fef0d5755" exitCode=0 Nov 04 05:27:42.604820 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.604799 5104 manager.go:324] Recovery completed Nov 04 05:27:42.614835 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.614813 5104 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:27:42.620440 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.620425 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.622528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.622508 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.622608 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.622541 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.622608 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.622554 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.625053 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625035 5104 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:27:42.625053 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625050 5104 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:27:42.625053 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625053 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.625249 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625068 5104 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:27:42.625249 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625230 5104 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625246 5104 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625272 5104 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625277 5104 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625283 5104 policy_none.go:50] "Start" Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625298 5104 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:27:42.625314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625309 5104 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:27:42.625472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625457 5104 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 05:27:42.625472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625466 5104 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 05:27:42.625472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625472 5104 policy_none.go:44] "Start" Nov 04 05:27:42.625911 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625898 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.625959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625923 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.625959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.625932 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.630940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.630874 5104 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:27:42.633477 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633463 5104 manager.go:341] "Starting Device Plugin manager" Nov 04 05:27:42.633582 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.633487 5104 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:27:42.633582 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633496 5104 server.go:87] "Starting device plugin registration server" Nov 04 05:27:42.633757 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633745 5104 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:27:42.633821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633756 5104 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:27:42.633876 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633832 5104 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:27:42.633947 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633917 5104 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:27:42.634042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633955 5104 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:27:42.634042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.633962 5104 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:27:42.634139 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.634009 5104 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:27:42.633941578Z","UUID":"e0f0a449-6433-4314-9f8d-feb917c3b7e8","Handler":null,"Name":"","Endpoint":""} Nov 04 05:27:42.635544 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.635525 5104 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:27:42.635646 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.635559 5104 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" Nov 04 05:27:42.635646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.635560 5104 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:27:42.635646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.635604 5104 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:27:42.642795 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.642384 5104 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal"] Nov 04 05:27:42.642908 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.642893 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.643776 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.643759 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.643899 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.643781 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.643899 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.643791 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.644059 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.644046 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.644327 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.644299 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.645194 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645178 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.645309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645204 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.645309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645232 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.645309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645179 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.645309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645279 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.645309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645300 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.645521 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.645397 5104 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.645687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645650 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"fd7b8286909c543935667f4273478b4f","Type":"ContainerStarted","Data":"ceb796cd7702a12695dfb6d995893868ceb535f79d97748571213df40203b9c8"} Nov 04 05:27:42.645806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645698 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"fd7b8286909c543935667f4273478b4f","Type":"ContainerStarted","Data":"ecf5ba14a75f7fdf438510c3f844489f692bb8838345b87b046942c60778ebea"} Nov 04 05:27:42.645806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645728 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerStarted","Data":"31bdcfe0fd2c5e059fe7d60afdc790782d06c32b375e5e736c9499baae46a60b"} Nov 04 05:27:42.645806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645743 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.645806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645749 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerDied","Data":"2138b6be6e2b976bc86e8ba4704124ba825d545a86f79751ee8c809fef0d5755"} Nov 04 05:27:42.645806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.645769 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" event={"ID":"63605d34945999c60632e953468d1ece","Type":"ContainerStarted","Data":"8967714b6327d8c4f4d560dd1029fc7abd0f16d6bc9852633d7120f7005fd8e2"} Nov 04 05:27:42.646535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.646518 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.646649 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.646550 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.646649 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.646580 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.646800 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.646746 5104 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-34.us-west-2.compute.internal\" not found" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.718945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.718926 5104 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 05:27:42.719698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.719680 5104 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:42.719904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.719886 5104 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:42.733883 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.733867 5104 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:42.734791 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.734776 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:42.734860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.734806 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:42.734860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.734816 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:42.734860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.734851 5104 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.770086 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.770038 5104 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:42.814515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.814492 5104 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.819671 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.819649 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.819769 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.819684 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.819769 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.819715 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.821015 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.820996 5104 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:42.877060 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.877039 5104 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.877160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.877097 5104 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.877763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.877739 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeNotReady" Nov 04 05:27:42.877861 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.877776 5104 setters.go:546] "Node became not ready" node="ip-10-0-139-34.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T05:27:42Z","lastTransitionTime":"2026-11-04T05:27:42Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"} Nov 04 05:27:42.878472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.878459 5104 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:42.878526 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.878494 5104 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.878526 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.878505 5104 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920107 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920134 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920153 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920200 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920241 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/63605d34945999c60632e953468d1ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"63605d34945999c60632e953468d1ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.920361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.920245 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd7b8286909c543935667f4273478b4f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" (UID: \"fd7b8286909c543935667f4273478b4f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:42.936727 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.936707 5104 nodeinfomanager.go:471] existing CSINode "ip-10-0-139-34.us-west-2.compute.internal" is owned by different node (oldNodeID="585c726a-ffac-46dd-802b-319e8d080b88", newNodeID=""), cleaning up... Nov 04 05:27:42.976998 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:42.976976 5104 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:42.977092 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:42.977016 5104 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:43.500596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.500569 5104 apiserver.go:52] "Watching apiserver" Nov 04 05:27:43.595040 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.594983 5104 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:43.595698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.595428 5104 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:43.657020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.656973 5104 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:43.657020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.657009 5104 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:43.657020 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:43.657034 5104 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:43.657020 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:27:43.657052 5104 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal" Nov 04 05:27:43.657486 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.657284 5104 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:27:43.657882 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.657839 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-ktrgr","openshift-insights/insights-runtime-extractor-8hnv7","openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7","openshift-dns/dns-default-c4xxf","openshift-dns/node-resolver-9lljt","openshift-multus/multus-rnjxj","openshift-monitoring/prometheus-operator-9cd89b74c-cw92n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-34.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p","kube-system/konnectivity-agent-qgzd8","openshift-cluster-node-tuning-operator/tuned-cfkjk","openshift-ingress-canary/ingress-canary-scktk","openshift-network-operator/iptables-alerter-crv6l","kube-system/global-pull-secret-syncer-dz6r4","openshift-network-diagnostics/network-check-target-fb7vr","openshift-multus/multus-additional-cni-plugins-zmrbn","kube-system/kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-vmjtb","openshift-service-ca/service-ca-5fffc97bdf-8klfn","openshift-console/console-85665dc646-2lvcs","openshift-multus/network-metrics-daemon-swb4f"] Nov 04 05:27:43.665791 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.665771 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tzr9z"] Nov 04 05:27:43.670326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.670309 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.712289 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.712263 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.715662 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.715645 5104 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:27:43.725438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725413 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.725522 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725440 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.725522 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725456 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.725522 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725474 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.725650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725582 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.725840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725717 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.725955 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725875 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.726013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725958 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:27:43.726013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.725988 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.726013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726009 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:27:43.726140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726025 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:27:43.726140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726108 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:43.726140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726107 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.726140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726106 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726154 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726202 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726234 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726247 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726252 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726250 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726277 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726300 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:27:43.726317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726319 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726316 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726378 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726395 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6725d36-ff4f-4770-a367-6bb72b84c8a6-tmp-dir\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726426 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726452 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726545 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:27:43.726684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726610 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:27:43.726889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726802 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:27:43.726889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726878 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:43.726957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726910 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4747d\" (UniqueName: \"kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:43.726957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726928 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.727042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.726975 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-sys\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.727042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727024 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-tls\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.727121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727050 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz5v9\" (UniqueName: \"kubernetes.io/projected/64dba94c-196c-42cc-953a-5fc0864b7614-kube-api-access-jz5v9\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.727121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727067 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.727121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727106 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fs5\" (UniqueName: \"kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.727121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727117 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4720d09-b1ed-4387-bc2f-5250e4bef23c-metrics-certs\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:43.727297 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727131 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.727297 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727163 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqkfr\" (UniqueName: \"kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.727297 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727279 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.727416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727309 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-metrics-client-ca\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.727416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727335 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.727416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727368 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slpp\" (UniqueName: \"kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.727416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727412 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.727583 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727569 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.727626 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727602 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.727682 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727656 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.727682 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727666 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-data-volume\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.727763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727683 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.727763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727707 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.727763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727734 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.727763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727752 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.727910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727767 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-tmp\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.727910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727781 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgc4h\" (UniqueName: \"kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.727910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727806 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.727910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727862 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.727910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727894 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvwwt\" (UniqueName: \"kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727929 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-tuned\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727952 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8d2\" (UniqueName: \"kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727970 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2qb\" (UniqueName: \"kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.727987 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728002 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.728085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728017 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzm88\" (UniqueName: \"kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728173 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728191 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728207 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728235 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728277 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.728341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728328 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728349 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-root\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728399 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728431 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728456 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728481 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728526 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728562 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728585 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.728597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728600 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728614 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728648 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728694 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728721 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-original-pull-secret\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728739 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-wtmp\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728759 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728777 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fh8g\" (UniqueName: \"kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728794 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728808 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728853 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728899 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-agent-certs\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728937 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728975 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.728992 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.729021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729007 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729060 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729080 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729102 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729128 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729148 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729166 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729180 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729196 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdd4t\" (UniqueName: \"kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729211 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729253 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-textfile\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729311 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729366 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pscgt\" (UniqueName: \"kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729407 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729498 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.729647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729536 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mgrt\" (UniqueName: \"kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729888 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729942 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.729977 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbj9\" (UniqueName: \"kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9\") pod \"migrator-66df5d496b-ckpv7\" (UID: \"1d754d22-453a-4115-9b0e-feb477099353\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730009 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730034 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730101 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730149 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730189 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730243 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.730298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730274 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730307 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730334 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730364 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730402 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730443 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730636 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.730745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730701 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730786 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730824 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730887 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730929 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hq46\" (UniqueName: \"kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730959 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.731058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.730996 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.731338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731086 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.731338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731121 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.731338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731154 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:43.731338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731184 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.731511 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731338 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985a8caa-672c-481d-95bb-3c1784c85236-tmp-dir\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.731709 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.731680 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovn-node-metrics-cert\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.759068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759016 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:27:43.759248 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759218 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.759334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759321 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:27:43.759380 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759330 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:27:43.759429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759376 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.759429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759415 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:27:43.759429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759415 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759452 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759455 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759493 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759505 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759509 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759522 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.759577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759567 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.759830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759590 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.759830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759604 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.759830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759642 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-cni-binary-copy\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.759830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759718 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.759830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759808 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:27:43.759989 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759839 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.759989 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.759879 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.760877 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.760861 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-env-overrides\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.762349 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.762332 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/298a3342-66d3-480a-9b7c-fd34a4dad76b-serviceca\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.763426 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763410 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.763506 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763443 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Nov 04 05:27:43.763506 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763481 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763575 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763517 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763626 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.763693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763642 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.763693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763659 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.763693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763681 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763661 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763740 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763785 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763808 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763812 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763826 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763843 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763850 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.763869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763869 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763898 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763917 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763938 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763942 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.763962 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:27:43.764140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.764000 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.764920 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.764906 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.764954 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.764944 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 05:27:43.765037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.765026 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:27:43.766242 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.766213 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/985a8caa-672c-481d-95bb-3c1784c85236-config-volume\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.766315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.766303 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535c0ff7-d319-49dd-a9f1-39b2179b9a1b-konnectivity-ca\") pod \"konnectivity-agent-qgzd8\" (UID: \"535c0ff7-d319-49dd-a9f1-39b2179b9a1b\") " pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:27:43.766449 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.766434 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.766535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.766519 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.767946 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.767931 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.767994 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.767935 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ab73326-38e7-4426-a211-20e190306dc6-multus-daemon-config\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.767994 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.767958 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70b57d48-f09f-4b30-ac48-63e558200561-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.769172 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.769153 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15462ea-de3a-472b-ab01-bc0a736b63d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.769255 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.769175 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.769255 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.769204 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-script-lib\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.769364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.769281 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1d1ed65-1711-4c55-b841-b97159f1d7af-ovnkube-config\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.769364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.769311 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.771450 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.771433 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d3c380e-88e6-4290-86f1-c554eda72ad5-iptables-alerter-script\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.771569 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.771558 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-cabundle\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.777099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777082 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 05:27:43.777337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777318 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 05:27:43.777337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777329 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 05:27:43.777468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777344 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Nov 04 05:27:43.777468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777441 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2ltvz\"" Nov 04 05:27:43.777574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777491 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:27:43.777574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777500 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:27:43.777574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777562 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:27:43.777574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777570 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777604 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777614 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sxrvr\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777631 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777650 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777658 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-szc85\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777666 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777661 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777666 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777706 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777714 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t7km8\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777721 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sxw5c\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777707 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:27:43.777761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.777752 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:27:43.779064 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.779046 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/985a8caa-672c-481d-95bb-3c1784c85236-metrics-tls\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.779124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.779111 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.779188 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.779173 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d71858-ad31-49c4-a7a8-279e5c15e82d-cert\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:43.780761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.780743 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.781896 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.781879 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.786054 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.786035 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70b57d48-f09f-4b30-ac48-63e558200561-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.787198 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.787184 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69feb234-dfe8-438d-a0c8-f26e159f0cbf-signing-key\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.793480 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.793459 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fh8g\" (UniqueName: \"kubernetes.io/projected/69feb234-dfe8-438d-a0c8-f26e159f0cbf-kube-api-access-7fh8g\") pod \"service-ca-5fffc97bdf-8klfn\" (UID: \"69feb234-dfe8-438d-a0c8-f26e159f0cbf\") " pod="openshift-service-ca/service-ca-5fffc97bdf-8klfn" Nov 04 05:27:43.794888 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.794871 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fs5\" (UniqueName: \"kubernetes.io/projected/34f11402-cfca-4c16-bc59-c4255e791348-kube-api-access-52fs5\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.795790 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.795771 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pscgt\" (UniqueName: \"kubernetes.io/projected/985a8caa-672c-481d-95bb-3c1784c85236-kube-api-access-pscgt\") pod \"dns-default-c4xxf\" (UID: \"985a8caa-672c-481d-95bb-3c1784c85236\") " pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:43.797149 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.797120 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"console-85665dc646-2lvcs\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:43.797430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.797412 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4747d\" (UniqueName: \"kubernetes.io/projected/d4720d09-b1ed-4387-bc2f-5250e4bef23c-kube-api-access-4747d\") pod \"network-metrics-daemon-swb4f\" (UID: \"d4720d09-b1ed-4387-bc2f-5250e4bef23c\") " pod="openshift-multus/network-metrics-daemon-swb4f" Nov 04 05:27:43.797430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.797417 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgc4h\" (UniqueName: \"kubernetes.io/projected/a15462ea-de3a-472b-ab01-bc0a736b63d2-kube-api-access-kgc4h\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.797536 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.797449 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvwwt\" (UniqueName: \"kubernetes.io/projected/d1d1ed65-1711-4c55-b841-b97159f1d7af-kube-api-access-lvwwt\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.797536 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.797528 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2slpp\" (UniqueName: \"kubernetes.io/projected/6d3c380e-88e6-4290-86f1-c554eda72ad5-kube-api-access-2slpp\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.799341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.799324 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mgrt\" (UniqueName: \"kubernetes.io/projected/5ab73326-38e7-4426-a211-20e190306dc6-kube-api-access-8mgrt\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.799958 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.799943 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2qb\" (UniqueName: \"kubernetes.io/projected/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-kube-api-access-qh2qb\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.801807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.801789 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzm88\" (UniqueName: \"kubernetes.io/projected/298a3342-66d3-480a-9b7c-fd34a4dad76b-kube-api-access-nzm88\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.802590 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.802575 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdd4t\" (UniqueName: \"kubernetes.io/projected/70b57d48-f09f-4b30-ac48-63e558200561-kube-api-access-sdd4t\") pod \"prometheus-operator-9cd89b74c-cw92n\" (UID: \"70b57d48-f09f-4b30-ac48-63e558200561\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-cw92n" Nov 04 05:27:43.810732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.810717 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8d2\" (UniqueName: \"kubernetes.io/projected/c6725d36-ff4f-4770-a367-6bb72b84c8a6-kube-api-access-hm8d2\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.810807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.810790 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbj9\" (UniqueName: \"kubernetes.io/projected/1d754d22-453a-4115-9b0e-feb477099353-kube-api-access-zmbj9\") pod \"migrator-66df5d496b-ckpv7\" (UID: \"1d754d22-453a-4115-9b0e-feb477099353\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ckpv7" Nov 04 05:27:43.823262 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.823214 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqkfr\" (UniqueName: \"kubernetes.io/projected/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-kube-api-access-gqkfr\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.826460 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.826444 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9fxp\" (UniqueName: \"kubernetes.io/projected/c7c36da8-7108-415e-a735-26d714bf56cd-kube-api-access-m9fxp\") pod \"network-check-target-fb7vr\" (UID: \"c7c36da8-7108-415e-a735-26d714bf56cd\") " pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:43.826864 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.826847 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hq46\" (UniqueName: \"kubernetes.io/projected/95d71858-ad31-49c4-a7a8-279e5c15e82d-kube-api-access-6hq46\") pod \"ingress-canary-scktk\" (UID: \"95d71858-ad31-49c4-a7a8-279e5c15e82d\") " pod="openshift-ingress-canary/ingress-canary-scktk" Nov 04 05:27:43.831640 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831616 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831643 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831659 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831675 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831697 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831720 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-ovn\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.831734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831731 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3c380e-88e6-4290-86f1-c554eda72ad5-host-slash\") pod \"iptables-alerter-crv6l\" (UID: \"6d3c380e-88e6-4290-86f1-c554eda72ad5\") " pod="openshift-network-operator/iptables-alerter-crv6l" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831723 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831756 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-multus-certs\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831765 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831779 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831804 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831817 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-sys-fs\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831817 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831828 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831861 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-run\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831864 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-root\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831872 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-sys\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831844 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-kubelet-config\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831888 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831905 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831909 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-root\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831919 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831933 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831938 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831948 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-socket-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831948 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831971 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831976 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-multus\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831987 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysconfig\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832013 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-hostroot\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832016 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.831988 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-conf-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832045 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832056 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832061 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832079 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-etc-kubernetes\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832088 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-wtmp\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832112 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832118 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54f8b1f3-a80d-47a0-bbb8-524762fdd08e-dbus\") pod \"global-pull-secret-syncer-dz6r4\" (UID: \"54f8b1f3-a80d-47a0-bbb8-524762fdd08e\") " pod="kube-system/global-pull-secret-syncer-dz6r4" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832138 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832167 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832186 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832203 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832213 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-wtmp\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.832185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832217 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832256 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832266 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6725d36-ff4f-4770-a367-6bb72b84c8a6-hosts-file\") pod \"node-resolver-9lljt\" (UID: \"c6725d36-ff4f-4770-a367-6bb72b84c8a6\") " pod="openshift-dns/node-resolver-9lljt" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832260 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-var-lib-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832281 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-lib-modules\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832288 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832288 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-etc-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832295 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-device-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832304 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-openvswitch\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832312 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-var-lib-kubelet\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832321 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-textfile\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832333 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-kubelet\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832340 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832485 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832516 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832540 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832560 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832565 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-cnibin\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832582 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832599 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-slash\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832604 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832591 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/298a3342-66d3-480a-9b7c-fd34a4dad76b-host\") pod \"node-ca-ktrgr\" (UID: \"298a3342-66d3-480a-9b7c-fd34a4dad76b\") " pod="openshift-image-registry/node-ca-ktrgr" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832632 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832654 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-run-netns\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832651 5104 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832684 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832689 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-system-cni-dir\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832696 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-textfile\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832654 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-log-socket\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832698 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-sysctl-conf\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832696 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-modprobe-d\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832738 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832753 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832792 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-run-systemd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832794 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-node-log\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832773 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832831 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832841 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832858 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832874 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832891 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832896 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-host\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832908 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832930 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-var-lib-cni-bin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832937 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-kubernetes\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832938 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832972 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832973 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833013 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc2d36d-fca3-4540-a8e8-27ea66fea7e3-crio-socket\") pod \"insights-runtime-extractor-8hnv7\" (UID: \"edc2d36d-fca3-4540-a8e8-27ea66fea7e3\") " pod="openshift-insights/insights-runtime-extractor-8hnv7" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.832970 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-netd\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833032 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833057 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833071 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-etc-selinux\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833086 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833111 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833117 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0d8aa45-4cab-40ab-81f8-0f60ec89009b-etc-systemd\") pod \"tuned-cfkjk\" (UID: \"f0d8aa45-4cab-40ab-81f8-0f60ec89009b\") " pod="openshift-cluster-node-tuning-operator/tuned-cfkjk" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833136 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833158 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833163 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-cnibin\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833178 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-multus-socket-dir-parent\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833186 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833212 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34f11402-cfca-4c16-bc59-c4255e791348-registration-dir\") pod \"aws-ebs-csi-driver-node-dr79p\" (UID: \"34f11402-cfca-4c16-bc59-c4255e791348\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dr79p" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833234 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15462ea-de3a-472b-ab01-bc0a736b63d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmrbn\" (UID: \"a15462ea-de3a-472b-ab01-bc0a736b63d2\") " pod="openshift-multus/multus-additional-cni-plugins-zmrbn" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833243 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833249 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-k8s-cni-cncf-io\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833274 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833278 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-systemd-units\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833289 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-sys\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833313 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-tls\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833313 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ab73326-38e7-4426-a211-20e190306dc6-host-run-netns\") pod \"multus-rnjxj\" (UID: \"5ab73326-38e7-4426-a211-20e190306dc6\") " pod="openshift-multus/multus-rnjxj" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833329 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz5v9\" (UniqueName: \"kubernetes.io/projected/64dba94c-196c-42cc-953a-5fc0864b7614-kube-api-access-jz5v9\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833350 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833357 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64dba94c-196c-42cc-953a-5fc0864b7614-sys\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833389 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-metrics-client-ca\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833408 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833413 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833452 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-kubelet\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.833681 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833462 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1d1ed65-1711-4c55-b841-b97159f1d7af-host-cni-bin\") pod \"ovnkube-node-vmjtb\" (UID: \"d1d1ed65-1711-4c55-b841-b97159f1d7af\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:43.835869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.833878 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64dba94c-196c-42cc-953a-5fc0864b7614-metrics-client-ca\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.835869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.834859 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.835869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.835350 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64dba94c-196c-42cc-953a-5fc0864b7614-node-exporter-tls\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.896579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.896547 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz5v9\" (UniqueName: \"kubernetes.io/projected/64dba94c-196c-42cc-953a-5fc0864b7614-kube-api-access-jz5v9\") pod \"node-exporter-tzr9z\" (UID: \"64dba94c-196c-42cc-953a-5fc0864b7614\") " pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:43.935048 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.935021 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:43.985326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:43.985291 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tzr9z" Nov 04 05:27:44.005170 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:44.005151 5104 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:27:44.144589 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:44.144510 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:44.357185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:44.357155 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:44.600582 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:44.600497 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzr9z" event={"ID":"64dba94c-196c-42cc-953a-5fc0864b7614","Type":"ContainerStarted","Data":"1fe5012a4676399d2940a82fdd3a24355a018dbea2c74b6563c390602a8035b7"} Nov 04 05:27:45.104589 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:45.104549 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:45.147023 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:45.146995 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c4xxf" Nov 04 05:27:45.347939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:45.347908 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:46.019256 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:46.019211 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:46.607812 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:46.607775 5104 generic.go:356] "Generic (PLEG): container finished" podID="64dba94c-196c-42cc-953a-5fc0864b7614" containerID="3fb31132746139dde70a3fd77921519064d22de451f951ccdf2b0be0494ac834" exitCode=0 Nov 04 05:27:46.607969 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:46.607858 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzr9z" event={"ID":"64dba94c-196c-42cc-953a-5fc0864b7614","Type":"ContainerDied","Data":"3fb31132746139dde70a3fd77921519064d22de451f951ccdf2b0be0494ac834"} Nov 04 05:27:47.072125 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.072100 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:47.367101 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.367022 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:47.591189 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.591162 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:47.612645 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.612614 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzr9z" event={"ID":"64dba94c-196c-42cc-953a-5fc0864b7614","Type":"ContainerStarted","Data":"5db2fcce5ebc3205a0c27e43a025fa3181bc3e7646d3eac6c42f58273c4dba7b"} Nov 04 05:27:47.612783 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.612669 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzr9z" event={"ID":"64dba94c-196c-42cc-953a-5fc0864b7614","Type":"ContainerStarted","Data":"6d288cf9b789644289e2a456c9ca86562e5b0f0e59f9eed8ea94942c8029f15d"} Nov 04 05:27:47.804603 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:47.804574 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:48.012105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:48.012075 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:48.223172 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:48.223144 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:48.430625 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:48.430578 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:48.637555 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:48.637476 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:49.203773 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.203742 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:49.507858 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.507828 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:27:49.508326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.508036 5104 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:49.858395 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.858321 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:49.959748 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.959715 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:49.981903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:49.981860 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tzr9z" podStartSLOduration=5.060051118 podStartE2EDuration="6.98184663s" totalImagesPullingTime="1.853136563s" totalInitContainerRuntime="68.658949ms" isStatefulPod=true podCreationTimestamp="2026-11-04 05:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:47.736252967 +0000 UTC m=+5.300636262" watchObservedRunningTime="2026-11-04 05:27:49.98184663 +0000 UTC m=+7.546229931" Nov 04 05:27:50.189347 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:50.189257 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:51.091435 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:51.091336 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:51.299102 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:51.299067 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:51.845058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:51.845020 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:27:52.360267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:52.360215 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:52.570579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:52.570542 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:52.798179 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:52.798150 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:53.067723 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.067649 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:53.098168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.098139 5104 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-34.us-west-2.compute.internal" event="NodeReady" Nov 04 05:27:53.385883 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.385809 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:27:53.420986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.420960 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:27:53.423980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.423963 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.488711 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.488682 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:27:53.504298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504267 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504304 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504326 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504389 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvf6s\" (UniqueName: \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504510 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504452 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504510 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504477 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504510 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504497 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.504595 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.504530 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.543541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.543515 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 05:27:53.543986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.543967 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:27:53.605233 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605193 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605354 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605249 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605410 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605361 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605406 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605446 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605464 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605483 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605514 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvf6s\" (UniqueName: \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.605926 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.605897 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.606044 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.606029 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.606242 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.606203 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.606318 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.606239 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.606542 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.606524 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.607970 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.607949 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.608020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.607953 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.668938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.668867 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvf6s\" (UniqueName: \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\") pod \"console-654587d875-mvcj8\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:53.734055 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:53.734032 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:27:54.110798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:54.110767 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:27:54.114588 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:54.114559 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908041d4_f8fa_4ec8_ad29_ecc1ea01f99f.slice/crio-887481343b821e176b5b0cc2f52ff248bc5ac2162138026c6c746dc1896351b5 WatchSource:0}: Error finding container 887481343b821e176b5b0cc2f52ff248bc5ac2162138026c6c746dc1896351b5: Status 404 returned error can't find the container with id 887481343b821e176b5b0cc2f52ff248bc5ac2162138026c6c746dc1896351b5 Nov 04 05:27:54.638542 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:54.638506 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654587d875-mvcj8" event={"ID":"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f","Type":"ContainerStarted","Data":"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb"} Nov 04 05:27:54.638960 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:54.638566 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654587d875-mvcj8" event={"ID":"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f","Type":"ContainerStarted","Data":"887481343b821e176b5b0cc2f52ff248bc5ac2162138026c6c746dc1896351b5"} Nov 04 05:27:54.765237 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:54.765181 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-654587d875-mvcj8" podStartSLOduration=1.765169234 podStartE2EDuration="1.765169234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:54.76462386 +0000 UTC m=+12.329007158" watchObservedRunningTime="2026-11-04 05:27:54.765169234 +0000 UTC m=+12.329552527" Nov 04 05:27:56.813060 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:56.813027 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:57.022202 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:57.022156 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:57.250115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:57.250059 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:57.456285 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:57.456231 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:58.081085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:58.081043 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:58.280863 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:58.280816 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:59.016608 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.016576 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:27:59.020922 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.020905 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.087926 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.087899 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:27:59.147725 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147693 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.147725 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147724 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.147939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147753 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.147939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147808 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.147939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147921 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x8mb\" (UniqueName: \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.148063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147961 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.148063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.147994 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.148063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.148014 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.248987 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.248958 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.248987 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.248986 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249010 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249025 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249042 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x8mb\" (UniqueName: \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249108 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249167 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249482 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249556 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.249990 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.249966 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.250097 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.250043 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.250290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.250273 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.250351 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.250335 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.252449 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.252429 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.252503 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.252437 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.311315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.311253 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x8mb\" (UniqueName: \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\") pod \"console-85c89fd4b9-q95q9\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.332738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.332712 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:27:59.653536 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.653450 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:27:59.705937 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.705908 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:27:59.738164 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:27:59.738133 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ca7a34_e492_45af_a382_b5099bf15b33.slice/crio-cc61f6112349b4563b833da1e8520af373360c39ec938d0098010f4e6f862eae WatchSource:0}: Error finding container cc61f6112349b4563b833da1e8520af373360c39ec938d0098010f4e6f862eae: Status 404 returned error can't find the container with id cc61f6112349b4563b833da1e8520af373360c39ec938d0098010f4e6f862eae Nov 04 05:27:59.925293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:27:59.925191 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:00.140180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.140144 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:00.367149 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.367119 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:00.376292 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.376263 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmjtb" Nov 04 05:28:00.489967 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:28:00.489934 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:28:00.658767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.658681 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c89fd4b9-q95q9" event={"ID":"61ca7a34-e492-45af-a382-b5099bf15b33","Type":"ContainerStarted","Data":"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735"} Nov 04 05:28:00.658767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.658735 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c89fd4b9-q95q9" event={"ID":"61ca7a34-e492-45af-a382-b5099bf15b33","Type":"ContainerStarted","Data":"cc61f6112349b4563b833da1e8520af373360c39ec938d0098010f4e6f862eae"} Nov 04 05:28:00.721178 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.721152 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qgzd8" Nov 04 05:28:00.779799 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:00.779750 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-85c89fd4b9-q95q9" podStartSLOduration=2.779737435 podStartE2EDuration="2.779737435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:00.779337762 +0000 UTC m=+18.343721051" watchObservedRunningTime="2026-11-04 05:28:00.779737435 +0000 UTC m=+18.344120734" Nov 04 05:28:01.264184 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:01.264143 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:01.493795 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:01.493758 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:02.282353 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:02.282320 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:02.485892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:02.485843 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:03.662984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:03.662955 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:03.734975 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:03.734946 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:03.735147 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:03.735001 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:03.739532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:03.739510 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:03.862470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:03.862433 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:04.675088 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:04.675062 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:06.603051 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:06.603020 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:09.041007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.040969 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:09.247558 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.247529 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:09.333798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.333723 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:28:09.333798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.333783 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:28:09.338105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.338085 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:28:09.696793 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.696712 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:28:09.847009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:09.846971 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:28:11.848291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:11.848265 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fb7vr" Nov 04 05:28:17.282142 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:17.282108 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:17.484041 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:17.484008 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:18.405111 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.405063 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-85665dc646-2lvcs" podUID="1148a311-091d-4817-98d0-9d6c4575acdf" containerName="console" containerID="cri-o://802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe" gracePeriod=15 Nov 04 05:28:18.652126 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.652102 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:28:18.720515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.720488 5104 generic.go:356] "Generic (PLEG): container finished" podID="1148a311-091d-4817-98d0-9d6c4575acdf" containerID="802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe" exitCode=0 Nov 04 05:28:18.720673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.720531 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85665dc646-2lvcs" Nov 04 05:28:18.720673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.720549 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85665dc646-2lvcs" event={"ID":"1148a311-091d-4817-98d0-9d6c4575acdf","Type":"ContainerDied","Data":"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe"} Nov 04 05:28:18.720673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.720589 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85665dc646-2lvcs" event={"ID":"1148a311-091d-4817-98d0-9d6c4575acdf","Type":"ContainerDied","Data":"dd807e34f40dd254412682c01c88fdc0068812768bdf9759e743be6096c1f8ac"} Nov 04 05:28:18.720673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.720609 5104 scope.go:122] "RemoveContainer" containerID="802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe" Nov 04 05:28:18.728153 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.728127 5104 scope.go:122] "RemoveContainer" containerID="802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe" Nov 04 05:28:18.728436 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:28:18.728415 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe\": container with ID starting with 802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe not found: ID does not exist" containerID="802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe" Nov 04 05:28:18.728513 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.728451 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe"} err="failed to get container status \"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe\": rpc error: code = NotFound desc = could not find container \"802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe\": container with ID starting with 802fe2139e6e3a21b0e8715a420908dd0b185d008f1cf12406019e2719796ffe not found: ID does not exist" Nov 04 05:28:18.798182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798153 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798333 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798211 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798333 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798259 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798333 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798289 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798338 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798377 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798406 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") pod \"1148a311-091d-4817-98d0-9d6c4575acdf\" (UID: \"1148a311-091d-4817-98d0-9d6c4575acdf\") " Nov 04 05:28:18.798690 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798661 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:18.798690 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798678 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:18.798864 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798843 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:18.798966 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.798946 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:18.800602 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.800575 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:18.800685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.800618 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "kube-api-access-gqct5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:18.800685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.800635 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert" pod "1148a311-091d-4817-98d0-9d6c4575acdf" (UID: "1148a311-091d-4817-98d0-9d6c4575acdf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:18.899010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.898979 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqct5\" (UniqueName: \"kubernetes.io/projected/1148a311-091d-4817-98d0-9d6c4575acdf-kube-api-access-gqct5\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899006 5104 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1148a311-091d-4817-98d0-9d6c4575acdf-tmp\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899018 5104 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-console-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899027 5104 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899036 5104 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-oauth-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899044 5104 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1148a311-091d-4817-98d0-9d6c4575acdf-console-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:18.899168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:18.899052 5104 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1148a311-091d-4817-98d0-9d6c4575acdf-oauth-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:19.141400 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:19.141366 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:28:19.145067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:19.145044 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85665dc646-2lvcs"] Nov 04 05:28:20.547620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:20.547584 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1148a311-091d-4817-98d0-9d6c4575acdf" path="/var/lib/kubelet/pods/1148a311-091d-4817-98d0-9d6c4575acdf/volumes" Nov 04 05:28:22.113611 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:22.113580 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:22.325568 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:22.325528 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:30.489770 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:28:30.489729 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:28:32.298201 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:32.298168 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:32.492113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:32.492067 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:34.866006 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:34.865968 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-654587d875-mvcj8" podUID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" containerName="console" containerID="cri-o://820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb" gracePeriod=15 Nov 04 05:28:35.113298 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.113275 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:35.215352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215294 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215337 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215377 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215411 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215445 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215465 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215490 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215512 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\" (UniqueName: \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\") pod \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\" (UID: \"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f\") " Nov 04 05:28:35.215832 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215557 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:35.215832 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215703 5104 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-tmp\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.215832 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215795 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:35.215939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215849 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:35.215939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.215886 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:35.216030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.216008 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:35.217683 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.217653 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:35.217683 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.217686 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "kube-api-access-pvf6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:35.217819 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.217686 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config" pod "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" (UID: "908041d4-f8fa-4ec8-ad29-ecc1ea01f99f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:35.316675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316644 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvf6s\" (UniqueName: \"kubernetes.io/projected/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-kube-api-access-pvf6s\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316671 5104 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-oauth-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316681 5104 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316690 5104 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-trusted-ca-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316700 5104 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316709 5104 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-oauth-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.316881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.316717 5104 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f-console-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:35.779737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.779703 5104 generic.go:356] "Generic (PLEG): container finished" podID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" containerID="820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb" exitCode=0 Nov 04 05:28:35.779903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.779755 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654587d875-mvcj8" event={"ID":"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f","Type":"ContainerDied","Data":"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb"} Nov 04 05:28:35.779903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.779762 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654587d875-mvcj8" Nov 04 05:28:35.779903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.779794 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654587d875-mvcj8" event={"ID":"908041d4-f8fa-4ec8-ad29-ecc1ea01f99f","Type":"ContainerDied","Data":"887481343b821e176b5b0cc2f52ff248bc5ac2162138026c6c746dc1896351b5"} Nov 04 05:28:35.779903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.779814 5104 scope.go:122] "RemoveContainer" containerID="820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb" Nov 04 05:28:35.788860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.788838 5104 scope.go:122] "RemoveContainer" containerID="820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb" Nov 04 05:28:35.789130 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:28:35.789109 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb\": container with ID starting with 820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb not found: ID does not exist" containerID="820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb" Nov 04 05:28:35.789178 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.789140 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb"} err="failed to get container status \"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb\": rpc error: code = NotFound desc = could not find container \"820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb\": container with ID starting with 820460f9b06c2023d4c1587587138f5eeeb4c170cdfbe64a64ac8d02d56d73bb not found: ID does not exist" Nov 04 05:28:35.947283 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.947249 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:28:35.998620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:35.998594 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654587d875-mvcj8"] Nov 04 05:28:36.546808 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:36.546773 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" path="/var/lib/kubelet/pods/908041d4-f8fa-4ec8-ad29-ecc1ea01f99f/volumes" Nov 04 05:28:42.516889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:42.516858 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:42.770312 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:42.770214 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:43.419803 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.419767 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d4f9f6979-8mdb8"] Nov 04 05:28:43.420001 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.419990 5104 cpu_manager.go:414] "containerMap: removing container" podUID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" containerName="console" Nov 04 05:28:43.420047 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.420003 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" containerName="console" Nov 04 05:28:43.420047 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.420024 5104 cpu_manager.go:414] "containerMap: removing container" podUID="1148a311-091d-4817-98d0-9d6c4575acdf" containerName="console" Nov 04 05:28:43.420047 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.420029 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1148a311-091d-4817-98d0-9d6c4575acdf" containerName="console" Nov 04 05:28:43.420130 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.420067 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="1148a311-091d-4817-98d0-9d6c4575acdf" containerName="console" Nov 04 05:28:43.420130 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.420076 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="908041d4-f8fa-4ec8-ad29-ecc1ea01f99f" containerName="console" Nov 04 05:28:43.424262 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.424244 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.466912 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.466882 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-oauth-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.466912 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.466914 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-trusted-ca-bundle\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.466933 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d274f14-b7f2-41ed-9d00-3725578d07fb-tmp\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.467056 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.467074 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhkx\" (UniqueName: \"kubernetes.io/projected/9d274f14-b7f2-41ed-9d00-3725578d07fb-kube-api-access-mdhkx\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.467089 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.467126 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-oauth-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.467246 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.467160 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-service-ca\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.480108 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.480086 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4f9f6979-8mdb8"] Nov 04 05:28:43.567662 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567633 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.567662 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567666 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhkx\" (UniqueName: \"kubernetes.io/projected/9d274f14-b7f2-41ed-9d00-3725578d07fb-kube-api-access-mdhkx\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567683 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567700 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-oauth-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567734 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-service-ca\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567765 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-oauth-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567781 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-trusted-ca-bundle\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.567807 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d274f14-b7f2-41ed-9d00-3725578d07fb-tmp\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.568312 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d274f14-b7f2-41ed-9d00-3725578d07fb-tmp\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.568530 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.568555 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-service-ca\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.568607 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-oauth-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.568741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.568730 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d274f14-b7f2-41ed-9d00-3725578d07fb-trusted-ca-bundle\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.570283 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.570264 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-serving-cert\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.570380 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.570303 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d274f14-b7f2-41ed-9d00-3725578d07fb-console-oauth-config\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.631625 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.631596 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhkx\" (UniqueName: \"kubernetes.io/projected/9d274f14-b7f2-41ed-9d00-3725578d07fb-kube-api-access-mdhkx\") pod \"console-7d4f9f6979-8mdb8\" (UID: \"9d274f14-b7f2-41ed-9d00-3725578d07fb\") " pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.734844 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.734802 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:43.772279 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.772186 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:43.972651 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:43.972618 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:44.107808 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:44.107720 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4f9f6979-8mdb8"] Nov 04 05:28:44.138359 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:28:44.138330 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d274f14_b7f2_41ed_9d00_3725578d07fb.slice/crio-91993a35395c5e4341467f2053e4d6e966200b9221d8674369485008cc12934f WatchSource:0}: Error finding container 91993a35395c5e4341467f2053e4d6e966200b9221d8674369485008cc12934f: Status 404 returned error can't find the container with id 91993a35395c5e4341467f2053e4d6e966200b9221d8674369485008cc12934f Nov 04 05:28:44.182475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:44.182443 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:44.813654 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:44.813618 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4f9f6979-8mdb8" event={"ID":"9d274f14-b7f2-41ed-9d00-3725578d07fb","Type":"ContainerStarted","Data":"82da908d57323d39ff706ef52fb561fa2e936b8b0e5b3a5da1ce2be2f7e19d0e"} Nov 04 05:28:44.814129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:44.813669 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4f9f6979-8mdb8" event={"ID":"9d274f14-b7f2-41ed-9d00-3725578d07fb","Type":"ContainerStarted","Data":"91993a35395c5e4341467f2053e4d6e966200b9221d8674369485008cc12934f"} Nov 04 05:28:44.936977 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:44.936924 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7d4f9f6979-8mdb8" podStartSLOduration=1.936910642 podStartE2EDuration="1.936910642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:44.93654222 +0000 UTC m=+62.500925511" watchObservedRunningTime="2026-11-04 05:28:44.936910642 +0000 UTC m=+62.501293941" Nov 04 05:28:53.735496 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:53.735460 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:53.735975 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:53.735520 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:53.740290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:53.740265 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:53.849165 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:53.849139 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d4f9f6979-8mdb8" Nov 04 05:28:54.037981 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:54.037908 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:28:55.157734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:55.157699 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:28:55.360388 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:28:55.360353 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:00.487646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:00.487617 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:00.489448 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:29:00.489421 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:29:02.338037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:02.337991 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:02.560407 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:02.560374 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:19.060571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.060529 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-85c89fd4b9-q95q9" podUID="61ca7a34-e492-45af-a382-b5099bf15b33" containerName="console" containerID="cri-o://04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735" gracePeriod=15 Nov 04 05:29:19.297601 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.297578 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:29:19.412543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412449 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412498 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\" (UniqueName: \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412518 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412550 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412632 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412675 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\" (UniqueName: \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412705 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.412825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412821 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\") pod \"61ca7a34-e492-45af-a382-b5099bf15b33\" (UID: \"61ca7a34-e492-45af-a382-b5099bf15b33\") " Nov 04 05:29:19.413075 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.412954 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:29:19.413075 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.413041 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:19.413331 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.413305 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:19.413470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.413385 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:19.413540 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.413470 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:29:19.414724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.414697 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:19.414818 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.414744 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:29:19.414818 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.414763 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb" pod "61ca7a34-e492-45af-a382-b5099bf15b33" (UID: "61ca7a34-e492-45af-a382-b5099bf15b33"). InnerVolumeSpecName "kube-api-access-2x8mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:29:19.514291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514258 5104 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-trusted-ca-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514284 5104 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514293 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x8mb\" (UniqueName: \"kubernetes.io/projected/61ca7a34-e492-45af-a382-b5099bf15b33-kube-api-access-2x8mb\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514303 5104 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ca7a34-e492-45af-a382-b5099bf15b33-console-oauth-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514312 5104 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-console-config\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514320 5104 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514330 5104 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61ca7a34-e492-45af-a382-b5099bf15b33-tmp\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.514484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.514338 5104 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ca7a34-e492-45af-a382-b5099bf15b33-oauth-serving-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:29:19.927447 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.927413 5104 generic.go:356] "Generic (PLEG): container finished" podID="61ca7a34-e492-45af-a382-b5099bf15b33" containerID="04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735" exitCode=0 Nov 04 05:29:19.927619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.927458 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c89fd4b9-q95q9" Nov 04 05:29:19.927619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.927501 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c89fd4b9-q95q9" event={"ID":"61ca7a34-e492-45af-a382-b5099bf15b33","Type":"ContainerDied","Data":"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735"} Nov 04 05:29:19.927619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.927552 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c89fd4b9-q95q9" event={"ID":"61ca7a34-e492-45af-a382-b5099bf15b33","Type":"ContainerDied","Data":"cc61f6112349b4563b833da1e8520af373360c39ec938d0098010f4e6f862eae"} Nov 04 05:29:19.927619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.927572 5104 scope.go:122] "RemoveContainer" containerID="04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735" Nov 04 05:29:19.935499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.935481 5104 scope.go:122] "RemoveContainer" containerID="04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735" Nov 04 05:29:19.935747 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:29:19.935727 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735\": container with ID starting with 04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735 not found: ID does not exist" containerID="04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735" Nov 04 05:29:19.935810 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:19.935754 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735"} err="failed to get container status \"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735\": rpc error: code = NotFound desc = could not find container \"04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735\": container with ID starting with 04623698032b8b49548156454cb5fe3dafe6fb57092f38aeb06398920c736735 not found: ID does not exist" Nov 04 05:29:20.098011 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:20.097978 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:29:20.149153 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:20.149126 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85c89fd4b9-q95q9"] Nov 04 05:29:20.547132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:20.547098 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61ca7a34-e492-45af-a382-b5099bf15b33" path="/var/lib/kubelet/pods/61ca7a34-e492-45af-a382-b5099bf15b33/volumes" Nov 04 05:29:21.320323 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:21.320277 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:30.489367 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:29:30.489327 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:29:43.571586 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:43.571551 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:29:43.804644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:29:43.804608 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:30:00.489438 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:30:00.489390 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:30:00.830135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:30:00.830055 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:30:30.488489 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:30:30.488451 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:31:00.488471 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:31:00.488396 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:31:01.199621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:01.199588 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:05.982031 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:05.981981 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:06.260241 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:06.260137 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:06.474978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:06.474942 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:07.441068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.441031 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg"] Nov 04 05:31:07.441478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.441300 5104 cpu_manager.go:414] "containerMap: removing container" podUID="61ca7a34-e492-45af-a382-b5099bf15b33" containerName="console" Nov 04 05:31:07.441478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.441311 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61ca7a34-e492-45af-a382-b5099bf15b33" containerName="console" Nov 04 05:31:07.441478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.441356 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="61ca7a34-e492-45af-a382-b5099bf15b33" containerName="console" Nov 04 05:31:07.444110 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.444094 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.501467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.501361 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg"] Nov 04 05:31:07.501761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.501738 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7"] Nov 04 05:31:07.505028 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.505013 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.510029 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510005 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gcjs\" (UniqueName: \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.510127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510044 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6fz4\" (UniqueName: \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.510173 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510150 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.510216 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510177 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.510312 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510291 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.510371 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.510329 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.552261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.552239 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 05:31:07.552374 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.552270 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 05:31:07.552374 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.552300 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-mzqq7\"" Nov 04 05:31:07.568505 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.568481 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-s992j\"" Nov 04 05:31:07.604067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.604048 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7"] Nov 04 05:31:07.611021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611000 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.611127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611028 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.611127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611059 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.611127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611074 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.611127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611106 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gcjs\" (UniqueName: \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.611127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611123 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6fz4\" (UniqueName: \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.611539 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611519 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.611607 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611546 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.611607 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611545 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.611767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.611750 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.674312 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.674288 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gcjs\" (UniqueName: \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.674416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.674383 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6fz4\" (UniqueName: \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:07.762778 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.762751 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:07.803979 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.803949 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:07.819509 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:07.819484 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:08.021202 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.021136 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:08.136622 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.136588 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg"] Nov 04 05:31:08.139816 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:08.139795 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881ef3a8_1afb_4706_9586_ec682afb0cf4.slice/crio-5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75 WatchSource:0}: Error finding container 5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75: Status 404 returned error can't find the container with id 5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75 Nov 04 05:31:08.227202 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.227176 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:08.250850 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.250821 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerStarted","Data":"5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75"} Nov 04 05:31:08.253630 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.253610 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7"] Nov 04 05:31:08.256627 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:08.256607 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb6320d_60c5_47aa_bc94_dd734b914c32.slice/crio-6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b WatchSource:0}: Error finding container 6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b: Status 404 returned error can't find the container with id 6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b Nov 04 05:31:08.444910 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:08.444840 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:09.225998 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.225971 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:09.255248 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.255204 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerStarted","Data":"6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b"} Nov 04 05:31:09.810998 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.810963 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww"] Nov 04 05:31:09.814577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.814554 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.825472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.825445 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.825592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.825562 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.825641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.825608 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvncs\" (UniqueName: \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.921046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.921012 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-l7n6v\"" Nov 04 05:31:09.921777 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.921697 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww"] Nov 04 05:31:09.926466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.926436 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.926588 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.926480 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvncs\" (UniqueName: \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.926588 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.926557 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.926959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.926888 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.926959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.926892 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:09.998786 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:09.998747 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvncs\" (UniqueName: \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:10.132916 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:10.132830 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:10.521055 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:10.521020 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww"] Nov 04 05:31:10.528729 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:10.528704 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3d0c5e_667e_4b84_a0bf_82a894120f55.slice/crio-27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da WatchSource:0}: Error finding container 27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da: Status 404 returned error can't find the container with id 27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da Nov 04 05:31:11.263024 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:11.262988 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerStarted","Data":"27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da"} Nov 04 05:31:12.408624 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.408592 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx"] Nov 04 05:31:12.412142 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.412123 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.444750 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.444719 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7gsz\" (UniqueName: \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.444867 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.444780 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.444867 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.444859 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.468058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.467997 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-4ndtr\"" Nov 04 05:31:12.468676 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.468621 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx"] Nov 04 05:31:12.512911 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.512885 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg"] Nov 04 05:31:12.516109 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.516081 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.545492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545467 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.545623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545517 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.545623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545545 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.545761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545629 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqwst\" (UniqueName: \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.545761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545734 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gsz\" (UniqueName: \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.545862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.545777 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.546091 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.546067 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.546183 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.546131 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.570044 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.570021 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-4gcmj\"" Nov 04 05:31:12.574591 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.574566 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg"] Nov 04 05:31:12.610643 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.610604 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7gsz\" (UniqueName: \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.612794 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.611640 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q"] Nov 04 05:31:12.615042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.615023 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.646921 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.646893 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhvlw\" (UniqueName: \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.647037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.646939 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqwst\" (UniqueName: \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.647110 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.647038 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.647171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.647074 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.647171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.647160 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.647834 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.647813 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.647925 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.647856 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.648213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.648186 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.670880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.670811 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-czqzx\"" Nov 04 05:31:12.671438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.671418 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q"] Nov 04 05:31:12.718071 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.710180 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqwst\" (UniqueName: \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.722456 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.722421 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8"] Nov 04 05:31:12.725967 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.725947 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.729659 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.729635 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:12.749238 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749194 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhvlw\" (UniqueName: \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.749360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749283 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.749360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749312 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.749524 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749360 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svxxb\" (UniqueName: \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.749524 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749397 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.749524 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749427 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.749805 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749783 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.749876 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.749804 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.781532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.780818 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-qcdvq\"" Nov 04 05:31:12.783072 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.782669 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8"] Nov 04 05:31:12.819185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.819155 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhvlw\" (UniqueName: \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:12.834405 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.834376 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:12.850893 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.850867 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svxxb\" (UniqueName: \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.851017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.850921 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.851017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.850949 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.851382 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.851359 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.851488 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.851468 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.912988 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.912956 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svxxb\" (UniqueName: \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:12.931749 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:12.931687 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:13.042980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.042949 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:13.127324 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.127246 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx"] Nov 04 05:31:13.235509 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.235476 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg"] Nov 04 05:31:13.239165 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:13.239126 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2120dba0_cdd4_4b0d_922d_e29dffda487c.slice/crio-3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056 WatchSource:0}: Error finding container 3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056: Status 404 returned error can't find the container with id 3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056 Nov 04 05:31:13.270478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.270443 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerStarted","Data":"3377a28e00dbc6d945354fd4bf6329db4bd3c62414a68355efcf7c60abdfe951"} Nov 04 05:31:13.271540 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.271516 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerStarted","Data":"3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056"} Nov 04 05:31:13.323390 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.323356 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q"] Nov 04 05:31:13.327960 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:13.327931 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00afd8a_366f_468a_9868_4e57b0f02fb5.slice/crio-8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a WatchSource:0}: Error finding container 8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a: Status 404 returned error can't find the container with id 8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a Nov 04 05:31:13.440266 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.440232 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8"] Nov 04 05:31:13.444607 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:13.444567 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e257f88_8761_4de8_a44b_af3039a98a28.slice/crio-c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb WatchSource:0}: Error finding container c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb: Status 404 returned error can't find the container with id c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb Nov 04 05:31:13.750693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.750662 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z"] Nov 04 05:31:13.755299 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.755272 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.810319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.810291 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z"] Nov 04 05:31:13.857847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.857810 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.858000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.857864 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.858000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.857946 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9t6\" (UniqueName: \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.959092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.959062 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9t6\" (UniqueName: \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.959284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.959113 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.959284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.959138 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.959590 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.959550 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:13.959746 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:13.959560 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:14.035098 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.034971 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9t6\" (UniqueName: \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:14.071243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.071006 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:14.277196 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.277156 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" event={"ID":"2e257f88-8761-4de8-a44b-af3039a98a28","Type":"ContainerStarted","Data":"c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb"} Nov 04 05:31:14.278719 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.278681 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerStarted","Data":"8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a"} Nov 04 05:31:14.471951 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.471918 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z"] Nov 04 05:31:14.504505 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:14.504455 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod326136e5_1f2e_4f13_902b_bd52964275a2.slice/crio-e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24 WatchSource:0}: Error finding container e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24: Status 404 returned error can't find the container with id e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24 Nov 04 05:31:14.729321 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.727557 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b"] Nov 04 05:31:14.732037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.731487 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.764779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.764578 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.764779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.764647 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.764779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.764689 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dllzb\" (UniqueName: \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.788743 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.788715 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b"] Nov 04 05:31:14.866801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.865832 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.866801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.865886 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.866801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.865922 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dllzb\" (UniqueName: \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.866801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.866517 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.866801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.866719 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.941960 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.941879 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dllzb\" (UniqueName: \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:14.974402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:14.974249 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:15.052476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.052118 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:15.240202 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.240163 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:15.289328 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.289292 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" event={"ID":"326136e5-1f2e-4f13-902b-bd52964275a2","Type":"ContainerStarted","Data":"e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24"} Nov 04 05:31:15.381866 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.381792 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8"] Nov 04 05:31:15.387117 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.386447 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.445543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.445476 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ztc4z\"" Nov 04 05:31:15.474686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.473326 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 05:31:15.474686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.473420 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 05:31:15.474686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.473480 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476047 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb571d04-7026-4946-83dc-b487e5231e5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476100 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476128 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476153 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476178 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb571d04-7026-4946-83dc-b487e5231e5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476204 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476266 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476292 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.476515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.476317 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztx4\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-kube-api-access-bztx4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.501570 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.501488 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8"] Nov 04 05:31:15.503782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.503756 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b"] Nov 04 05:31:15.550472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.550433 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:15.576792 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.576761 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb571d04-7026-4946-83dc-b487e5231e5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.576944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.576819 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577276 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577243 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577309 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577350 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577377 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb571d04-7026-4946-83dc-b487e5231e5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577547 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577403 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577547 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577470 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577547 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577517 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.577698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.577546 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztx4\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-kube-api-access-bztx4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.578173 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.578151 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.578809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.578785 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb571d04-7026-4946-83dc-b487e5231e5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.579332 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.579306 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.579584 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.579565 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.587321 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.587269 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb571d04-7026-4946-83dc-b487e5231e5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.601771 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.601700 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb571d04-7026-4946-83dc-b487e5231e5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.647829 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.647733 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.651718 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.651674 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztx4\" (UniqueName: \"kubernetes.io/projected/fb571d04-7026-4946-83dc-b487e5231e5a-kube-api-access-bztx4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-tmnh8\" (UID: \"fb571d04-7026-4946-83dc-b487e5231e5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.704589 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.704557 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:15.726551 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.723011 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf"] Nov 04 05:31:15.726551 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.726270 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.770546 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.770501 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779569 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779673 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779710 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779739 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779771 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779807 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779832 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np8gc\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-kube-api-access-np8gc\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779874 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.779986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.779909 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.788280 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.788240 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf"] Nov 04 05:31:15.814999 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.814967 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-fp2zd\"" Nov 04 05:31:15.881086 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881048 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881123 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881180 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881210 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881266 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881299 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881333 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881369 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np8gc\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-kube-api-access-np8gc\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.881490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.881408 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.882181 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.882155 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.883490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.883466 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.883759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.883735 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.884161 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.884137 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.884463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.884440 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.889819 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.889749 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.894158 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.894116 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.957670 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.957577 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np8gc\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-kube-api-access-np8gc\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.972968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.972905 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf\" (UID: \"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:15.981824 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:15.981761 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:16.046642 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:16.046356 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:16.158947 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:16.153343 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8"] Nov 04 05:31:16.300183 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:16.300084 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" event={"ID":"fb571d04-7026-4946-83dc-b487e5231e5a","Type":"ContainerStarted","Data":"c7b5dc55083f28938a84db25fdc6de5fcbd92d0c7c2bfd0c8f497a63addeb5a2"} Nov 04 05:31:16.302598 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:16.302530 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerStarted","Data":"4abc7e0d892825bd1a4598e8167f09f7a1fd2c69d0258eddad9c175a7a388af4"} Nov 04 05:31:16.504559 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:16.504461 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf"] Nov 04 05:31:17.319515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:17.319255 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" event={"ID":"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d","Type":"ContainerStarted","Data":"4d06dc82944cdc15eefc69a95866f1407dd4d581f615e54b2ac97a0397e22d9a"} Nov 04 05:31:19.448864 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:19.448682 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:27.470380 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.470343 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:27.470803 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.470434 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:27.470803 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.470484 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:27.474965 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.474932 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:27.475061 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.475011 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:27.475124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:27.475058 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:31:28.424240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.424191 5104 generic.go:356] "Generic (PLEG): container finished" podID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerID="40513fef2c49fa3eb334f6787e7c3b2be028fb636ad6d4f75db4808be5d6fd24" exitCode=0 Nov 04 05:31:28.424402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.424284 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerDied","Data":"40513fef2c49fa3eb334f6787e7c3b2be028fb636ad6d4f75db4808be5d6fd24"} Nov 04 05:31:28.428141 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.428113 5104 generic.go:356] "Generic (PLEG): container finished" podID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerID="8d3d48c43dab20158973b492696632b3414c852eddd65560ea1be2fe5eeff4f2" exitCode=0 Nov 04 05:31:28.428276 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.428197 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerDied","Data":"8d3d48c43dab20158973b492696632b3414c852eddd65560ea1be2fe5eeff4f2"} Nov 04 05:31:28.431327 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.431303 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" event={"ID":"fb571d04-7026-4946-83dc-b487e5231e5a","Type":"ContainerStarted","Data":"8a0bfe50827187ebcdaa9699c13abf16b0bfb6a83b587ec1abf1a3b41e31c45e"} Nov 04 05:31:28.433410 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.433389 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" event={"ID":"fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d","Type":"ContainerStarted","Data":"1b1514887d140b975838d926dca5e78c010892b6145e25159241f65a362de4af"} Nov 04 05:31:28.435911 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.435883 5104 generic.go:356] "Generic (PLEG): container finished" podID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerID="aaeb3afb5241482055e2143b0d8f9a45af4f861e81d477209b871de9a1c16a88" exitCode=0 Nov 04 05:31:28.436003 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.435986 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerDied","Data":"aaeb3afb5241482055e2143b0d8f9a45af4f861e81d477209b871de9a1c16a88"} Nov 04 05:31:28.439207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.439182 5104 generic.go:356] "Generic (PLEG): container finished" podID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerID="0691af0b920832082baa6fc8a9f761e0b8dafcf123ba8168b1cdb2fe65162f65" exitCode=0 Nov 04 05:31:28.439292 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.439267 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerDied","Data":"0691af0b920832082baa6fc8a9f761e0b8dafcf123ba8168b1cdb2fe65162f65"} Nov 04 05:31:28.442246 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.442071 5104 generic.go:356] "Generic (PLEG): container finished" podID="326136e5-1f2e-4f13-902b-bd52964275a2" containerID="d2e9b22a2f1102b3d749da6c0a10e725e2dcae44e2004d930b9491f32e993cff" exitCode=0 Nov 04 05:31:28.442246 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.442108 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" event={"ID":"326136e5-1f2e-4f13-902b-bd52964275a2","Type":"ContainerDied","Data":"d2e9b22a2f1102b3d749da6c0a10e725e2dcae44e2004d930b9491f32e993cff"} Nov 04 05:31:28.693571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.693370 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podStartSLOduration=2.388873295 podStartE2EDuration="13.693342829s" totalImagesPullingTime="11.304469534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:28.692487202 +0000 UTC m=+226.256870501" watchObservedRunningTime="2026-11-04 05:31:28.693342829 +0000 UTC m=+226.257726130" Nov 04 05:31:28.705679 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.705436 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:28.711075 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.711021 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:28.826592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:28.826489 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" podStartSLOduration=2.873352408 podStartE2EDuration="13.826471429s" totalImagesPullingTime="10.953119021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:28.826011751 +0000 UTC m=+226.390395053" watchObservedRunningTime="2026-11-04 05:31:28.826471429 +0000 UTC m=+226.390854732" Nov 04 05:31:29.047342 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.047298 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:29.053184 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.053156 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:29.450892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.450760 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:29.450892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.450814 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:29.451835 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.451780 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf" Nov 04 05:31:29.452170 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:29.452126 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:31:30.456342 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:30.456307 5104 generic.go:356] "Generic (PLEG): container finished" podID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerID="5b1dcb1ddf3f4663548c65b577439b39ff4dd9e71e0c116712cb3c75796d2dbc" exitCode=0 Nov 04 05:31:30.456772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:30.456452 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerDied","Data":"5b1dcb1ddf3f4663548c65b577439b39ff4dd9e71e0c116712cb3c75796d2dbc"} Nov 04 05:31:30.460992 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:30.460961 5104 generic.go:356] "Generic (PLEG): container finished" podID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerID="a8034aede04d79dd135e992f14e6e76f26a648008461f051ea064a99401ee725" exitCode=0 Nov 04 05:31:30.463492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:30.461963 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerDied","Data":"a8034aede04d79dd135e992f14e6e76f26a648008461f051ea064a99401ee725"} Nov 04 05:31:30.502938 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:31:30.502900 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:31:31.467005 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:31.466975 5104 generic.go:356] "Generic (PLEG): container finished" podID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerID="6c4c2f054c854e433ee20f5ba9852557dc1b78811830527672aff37ddc390f1c" exitCode=0 Nov 04 05:31:31.467520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:31.467326 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerDied","Data":"6c4c2f054c854e433ee20f5ba9852557dc1b78811830527672aff37ddc390f1c"} Nov 04 05:31:31.469238 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:31.469178 5104 generic.go:356] "Generic (PLEG): container finished" podID="3797da46-4333-454b-bbdb-73eabb06a68a" containerID="a23d432c5cc29c75830d44f70b2093b0935ad573e55561a6df79ae25906b48dd" exitCode=0 Nov 04 05:31:31.469515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:31.469485 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerDied","Data":"a23d432c5cc29c75830d44f70b2093b0935ad573e55561a6df79ae25906b48dd"} Nov 04 05:31:32.485123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:32.484068 5104 generic.go:356] "Generic (PLEG): container finished" podID="2e257f88-8761-4de8-a44b-af3039a98a28" containerID="d0d8928504f1505408457da2a3088a6a93e2e94e699c16c784edd0f4142ece2f" exitCode=0 Nov 04 05:31:32.485123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:32.484196 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" event={"ID":"2e257f88-8761-4de8-a44b-af3039a98a28","Type":"ContainerDied","Data":"d0d8928504f1505408457da2a3088a6a93e2e94e699c16c784edd0f4142ece2f"} Nov 04 05:31:32.488106 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:32.488080 5104 generic.go:356] "Generic (PLEG): container finished" podID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerID="0d571a0526fc8b5e6fbf97418997362e3e73695e9e1ce3cae30d83da0eccc925" exitCode=0 Nov 04 05:31:32.488208 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:32.488164 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerDied","Data":"0d571a0526fc8b5e6fbf97418997362e3e73695e9e1ce3cae30d83da0eccc925"} Nov 04 05:31:33.496305 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:33.496264 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerStarted","Data":"c9b49dee94d421342be07557752bccaa4df11e9d5c907fe655547bddeba9a803"} Nov 04 05:31:34.502693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:34.502658 5104 generic.go:356] "Generic (PLEG): container finished" podID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerID="c9b49dee94d421342be07557752bccaa4df11e9d5c907fe655547bddeba9a803" exitCode=0 Nov 04 05:31:34.503079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:34.502718 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerDied","Data":"c9b49dee94d421342be07557752bccaa4df11e9d5c907fe655547bddeba9a803"} Nov 04 05:31:35.509492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.509460 5104 generic.go:356] "Generic (PLEG): container finished" podID="2e257f88-8761-4de8-a44b-af3039a98a28" containerID="ec6d4616a6087c03b752a09a472ad369c99fdb81b44e629044c2d7f51fb311c4" exitCode=0 Nov 04 05:31:35.509939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.509550 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" event={"ID":"2e257f88-8761-4de8-a44b-af3039a98a28","Type":"ContainerDied","Data":"ec6d4616a6087c03b752a09a472ad369c99fdb81b44e629044c2d7f51fb311c4"} Nov 04 05:31:35.511505 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.511487 5104 generic.go:356] "Generic (PLEG): container finished" podID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerID="a2644f7fdf779d1b75dae235106570e5388b83088d7c38230ce2cbff636483b9" exitCode=0 Nov 04 05:31:35.511583 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.511519 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerDied","Data":"a2644f7fdf779d1b75dae235106570e5388b83088d7c38230ce2cbff636483b9"} Nov 04 05:31:35.513602 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.513585 5104 generic.go:356] "Generic (PLEG): container finished" podID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerID="b11c29f91fa02c25d274453804016a1a935093099ad6181d312f116a5d89a611" exitCode=0 Nov 04 05:31:35.513683 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.513662 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerDied","Data":"b11c29f91fa02c25d274453804016a1a935093099ad6181d312f116a5d89a611"} Nov 04 05:31:35.515436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.515397 5104 generic.go:356] "Generic (PLEG): container finished" podID="3797da46-4333-454b-bbdb-73eabb06a68a" containerID="341fdd7723dd67d285a5cf0b1f8108e94921bc39ae54e89a24cf095983e1802f" exitCode=0 Nov 04 05:31:35.515507 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.515453 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerDied","Data":"341fdd7723dd67d285a5cf0b1f8108e94921bc39ae54e89a24cf095983e1802f"} Nov 04 05:31:35.517341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.517323 5104 generic.go:356] "Generic (PLEG): container finished" podID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerID="c8b5df52cc002154b7c62b47a0746abe7ca2905b3f77766e459c166d61d50b7e" exitCode=0 Nov 04 05:31:35.517430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.517402 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerDied","Data":"c8b5df52cc002154b7c62b47a0746abe7ca2905b3f77766e459c166d61d50b7e"} Nov 04 05:31:35.519283 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.519199 5104 generic.go:356] "Generic (PLEG): container finished" podID="326136e5-1f2e-4f13-902b-bd52964275a2" containerID="f97ff883c5cca75e239f570bedb6d494064793067fde22b2088f3c4b95cf7ca3" exitCode=0 Nov 04 05:31:35.519283 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:35.519239 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" event={"ID":"326136e5-1f2e-4f13-902b-bd52964275a2","Type":"ContainerDied","Data":"f97ff883c5cca75e239f570bedb6d494064793067fde22b2088f3c4b95cf7ca3"} Nov 04 05:31:49.644307 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:49.643686 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerStarted","Data":"008bd785ad4520dc6e7d31eacbcbc38e1d58577d0df011e6d5d43e9998c0de20"} Nov 04 05:31:49.647992 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:49.647962 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerStarted","Data":"6e0e9fcdf07c8c34156fc8fc008ead9d140d7eadc3921bc7ebd738b6bfaf2e6f"} Nov 04 05:31:49.775460 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:49.775404 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" podStartSLOduration=2.7540844570000003 podStartE2EDuration="42.775386716s" totalImagesPullingTime="40.021302259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:49.774801187 +0000 UTC m=+247.339184487" watchObservedRunningTime="2026-11-04 05:31:49.775386716 +0000 UTC m=+247.339770017" Nov 04 05:31:50.665379 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:50.664757 5104 generic.go:356] "Generic (PLEG): container finished" podID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerID="6e0e9fcdf07c8c34156fc8fc008ead9d140d7eadc3921bc7ebd738b6bfaf2e6f" exitCode=0 Nov 04 05:31:50.665379 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:50.664900 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerDied","Data":"6e0e9fcdf07c8c34156fc8fc008ead9d140d7eadc3921bc7ebd738b6bfaf2e6f"} Nov 04 05:31:50.671095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:50.670821 5104 generic.go:356] "Generic (PLEG): container finished" podID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerID="008bd785ad4520dc6e7d31eacbcbc38e1d58577d0df011e6d5d43e9998c0de20" exitCode=0 Nov 04 05:31:50.671095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:50.670906 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerDied","Data":"008bd785ad4520dc6e7d31eacbcbc38e1d58577d0df011e6d5d43e9998c0de20"} Nov 04 05:31:51.922265 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:51.922047 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:51.936121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:51.935993 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.048938 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\") pod \"881ef3a8-1afb-4706-9586-ec682afb0cf4\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.048998 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\") pod \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.049032 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\") pod \"881ef3a8-1afb-4706-9586-ec682afb0cf4\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.049069 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\" (UniqueName: \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\") pod \"881ef3a8-1afb-4706-9586-ec682afb0cf4\" (UID: \"881ef3a8-1afb-4706-9586-ec682afb0cf4\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.049111 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\") pod \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.049149 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\" (UniqueName: \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\") pod \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\" (UID: \"a2fc1db1-9ad7-46a3-8761-eb072a877f05\") " Nov 04 05:31:52.050034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.049595 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle" pod "881ef3a8-1afb-4706-9586-ec682afb0cf4" (UID: "881ef3a8-1afb-4706-9586-ec682afb0cf4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:52.050641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.050570 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle" pod "a2fc1db1-9ad7-46a3-8761-eb072a877f05" (UID: "a2fc1db1-9ad7-46a3-8761-eb072a877f05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:52.058400 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.058375 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs" pod "881ef3a8-1afb-4706-9586-ec682afb0cf4" (UID: "881ef3a8-1afb-4706-9586-ec682afb0cf4"). InnerVolumeSpecName "kube-api-access-5gcjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:52.058985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.058957 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb" pod "a2fc1db1-9ad7-46a3-8761-eb072a877f05" (UID: "a2fc1db1-9ad7-46a3-8761-eb072a877f05"). InnerVolumeSpecName "kube-api-access-dllzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:52.064962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.064936 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util" pod "881ef3a8-1afb-4706-9586-ec682afb0cf4" (UID: "881ef3a8-1afb-4706-9586-ec682afb0cf4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:52.065191 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.065161 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util" pod "a2fc1db1-9ad7-46a3-8761-eb072a877f05" (UID: "a2fc1db1-9ad7-46a3-8761-eb072a877f05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.150952 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.150993 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.151008 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881ef3a8-1afb-4706-9586-ec682afb0cf4-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.151023 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gcjs\" (UniqueName: \"kubernetes.io/projected/881ef3a8-1afb-4706-9586-ec682afb0cf4-kube-api-access-5gcjs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.151039 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2fc1db1-9ad7-46a3-8761-eb072a877f05-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.151093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.151053 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dllzb\" (UniqueName: \"kubernetes.io/projected/a2fc1db1-9ad7-46a3-8761-eb072a877f05-kube-api-access-dllzb\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:52.693825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.693783 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" event={"ID":"a2fc1db1-9ad7-46a3-8761-eb072a877f05","Type":"ContainerDied","Data":"4abc7e0d892825bd1a4598e8167f09f7a1fd2c69d0258eddad9c175a7a388af4"} Nov 04 05:31:52.694007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.693872 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4abc7e0d892825bd1a4598e8167f09f7a1fd2c69d0258eddad9c175a7a388af4" Nov 04 05:31:52.694007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.693953 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r9j6b" Nov 04 05:31:52.702731 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.702702 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" event={"ID":"881ef3a8-1afb-4706-9586-ec682afb0cf4","Type":"ContainerDied","Data":"5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75"} Nov 04 05:31:52.702861 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.702751 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5debb3d39fb3609bc72fa80361a9878d031be45bc7798b593868804ad9310e75" Nov 04 05:31:52.702861 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.702828 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71flgbg" Nov 04 05:31:52.713641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.713605 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerStarted","Data":"27d05754d76d7262085f104c8e267a89b732384dadf871e0040129c2b10f57d3"} Nov 04 05:31:52.951757 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:52.951679 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" podStartSLOduration=4.341353925 podStartE2EDuration="43.951659956s" totalImagesPullingTime="39.610306031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:52.951393477 +0000 UTC m=+250.515776779" watchObservedRunningTime="2026-11-04 05:31:52.951659956 +0000 UTC m=+250.516043257" Nov 04 05:31:53.725056 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:53.725020 5104 generic.go:356] "Generic (PLEG): container finished" podID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerID="27d05754d76d7262085f104c8e267a89b732384dadf871e0040129c2b10f57d3" exitCode=0 Nov 04 05:31:53.725272 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:53.725134 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerDied","Data":"27d05754d76d7262085f104c8e267a89b732384dadf871e0040129c2b10f57d3"} Nov 04 05:31:54.730862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.730830 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerStarted","Data":"0615ef387a107d837fe81be0880ac3174f9aad2444bcb4230f737707dd420ca8"} Nov 04 05:31:54.733294 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.733259 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerStarted","Data":"d9ebf9d66cdc5d46ca5ba1c9f339ae3ddaa028907e8077bbef4871a19f435709"} Nov 04 05:31:54.735176 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.735151 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerStarted","Data":"db2936246d6bdf2c31a50dc2b4887650c1c1b212b9219bb1ff9f0faf3dcd6c39"} Nov 04 05:31:54.736985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.736961 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerStarted","Data":"49e0ea618e903be44e596a1965f0c27a7951e018d15c8211617414000355e34a"} Nov 04 05:31:54.738754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.738734 5104 generic.go:356] "Generic (PLEG): container finished" podID="326136e5-1f2e-4f13-902b-bd52964275a2" containerID="e15d1b117e8cb42138548b6b8065cb5dbe29d1ccacae8036abc82380d52ff665" exitCode=0 Nov 04 05:31:54.738842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.738811 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" event={"ID":"326136e5-1f2e-4f13-902b-bd52964275a2","Type":"ContainerDied","Data":"e15d1b117e8cb42138548b6b8065cb5dbe29d1ccacae8036abc82380d52ff665"} Nov 04 05:31:54.740504 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.740480 5104 generic.go:356] "Generic (PLEG): container finished" podID="2e257f88-8761-4de8-a44b-af3039a98a28" containerID="7da9abbc5de491b513b7641264df8e168f6cd9249878c596443540de3d4e0eb4" exitCode=0 Nov 04 05:31:54.740579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.740561 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" event={"ID":"2e257f88-8761-4de8-a44b-af3039a98a28","Type":"ContainerDied","Data":"7da9abbc5de491b513b7641264df8e168f6cd9249878c596443540de3d4e0eb4"} Nov 04 05:31:54.862680 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.862635 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" podStartSLOduration=3.087499744 podStartE2EDuration="42.862618837s" totalImagesPullingTime="39.775119093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:54.862414289 +0000 UTC m=+252.426797587" watchObservedRunningTime="2026-11-04 05:31:54.862618837 +0000 UTC m=+252.427002137" Nov 04 05:31:54.926430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.926406 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:54.993066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:54.993026 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" podStartSLOduration=2.835048066 podStartE2EDuration="42.99301361s" totalImagesPullingTime="40.157965544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:54.992067498 +0000 UTC m=+252.556450793" watchObservedRunningTime="2026-11-04 05:31:54.99301361 +0000 UTC m=+252.557396955" Nov 04 05:31:55.087405 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.087378 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\") pod \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " Nov 04 05:31:55.087528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.087421 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\" (UniqueName: \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\") pod \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " Nov 04 05:31:55.087528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.087458 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\") pod \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\" (UID: \"cb3d0c5e-667e-4b84-a0bf-82a894120f55\") " Nov 04 05:31:55.088056 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.088033 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle" pod "cb3d0c5e-667e-4b84-a0bf-82a894120f55" (UID: "cb3d0c5e-667e-4b84-a0bf-82a894120f55"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:55.089375 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.089338 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs" pod "cb3d0c5e-667e-4b84-a0bf-82a894120f55" (UID: "cb3d0c5e-667e-4b84-a0bf-82a894120f55"). InnerVolumeSpecName "kube-api-access-tvncs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:55.093140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.093115 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util" pod "cb3d0c5e-667e-4b84-a0bf-82a894120f55" (UID: "cb3d0c5e-667e-4b84-a0bf-82a894120f55"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:55.188289 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.188261 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:55.188289 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.188283 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb3d0c5e-667e-4b84-a0bf-82a894120f55-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:55.188289 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.188293 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvncs\" (UniqueName: \"kubernetes.io/projected/cb3d0c5e-667e-4b84-a0bf-82a894120f55-kube-api-access-tvncs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:55.257191 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.257137 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" podStartSLOduration=3.335344027 podStartE2EDuration="43.257123345s" totalImagesPullingTime="39.921779318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:55.256537484 +0000 UTC m=+252.820920784" watchObservedRunningTime="2026-11-04 05:31:55.257123345 +0000 UTC m=+252.821506643" Nov 04 05:31:55.521307 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.521259 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" podStartSLOduration=4.334146155 podStartE2EDuration="48.521245761s" totalImagesPullingTime="44.187099606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:31:55.389491613 +0000 UTC m=+252.953874912" watchObservedRunningTime="2026-11-04 05:31:55.521245761 +0000 UTC m=+253.085629100" Nov 04 05:31:55.746754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.746711 5104 generic.go:356] "Generic (PLEG): container finished" podID="3797da46-4333-454b-bbdb-73eabb06a68a" containerID="db2936246d6bdf2c31a50dc2b4887650c1c1b212b9219bb1ff9f0faf3dcd6c39" exitCode=0 Nov 04 05:31:55.747160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.746801 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerDied","Data":"db2936246d6bdf2c31a50dc2b4887650c1c1b212b9219bb1ff9f0faf3dcd6c39"} Nov 04 05:31:55.748887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.748856 5104 generic.go:356] "Generic (PLEG): container finished" podID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerID="49e0ea618e903be44e596a1965f0c27a7951e018d15c8211617414000355e34a" exitCode=0 Nov 04 05:31:55.749014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.748940 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerDied","Data":"49e0ea618e903be44e596a1965f0c27a7951e018d15c8211617414000355e34a"} Nov 04 05:31:55.750593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.750574 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" event={"ID":"cb3d0c5e-667e-4b84-a0bf-82a894120f55","Type":"ContainerDied","Data":"27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da"} Nov 04 05:31:55.750593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.750584 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cp7kww" Nov 04 05:31:55.750737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.750606 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27ca25929ff9e51face53ed2392e26fab90c120c92421d886186c469a6dbd6da" Nov 04 05:31:55.752693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.752672 5104 generic.go:356] "Generic (PLEG): container finished" podID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerID="0615ef387a107d837fe81be0880ac3174f9aad2444bcb4230f737707dd420ca8" exitCode=0 Nov 04 05:31:55.752816 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.752699 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerDied","Data":"0615ef387a107d837fe81be0880ac3174f9aad2444bcb4230f737707dd420ca8"} Nov 04 05:31:55.754738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.754719 5104 generic.go:356] "Generic (PLEG): container finished" podID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerID="d9ebf9d66cdc5d46ca5ba1c9f339ae3ddaa028907e8077bbef4871a19f435709" exitCode=0 Nov 04 05:31:55.754851 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.754800 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerDied","Data":"d9ebf9d66cdc5d46ca5ba1c9f339ae3ddaa028907e8077bbef4871a19f435709"} Nov 04 05:31:55.885902 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.885878 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:55.909732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.909711 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:55.995622 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.995588 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\") pod \"326136e5-1f2e-4f13-902b-bd52964275a2\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " Nov 04 05:31:55.995865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.995638 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\" (UniqueName: \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\") pod \"326136e5-1f2e-4f13-902b-bd52964275a2\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " Nov 04 05:31:55.995865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.995657 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\") pod \"326136e5-1f2e-4f13-902b-bd52964275a2\" (UID: \"326136e5-1f2e-4f13-902b-bd52964275a2\") " Nov 04 05:31:55.996073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.996049 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle" pod "326136e5-1f2e-4f13-902b-bd52964275a2" (UID: "326136e5-1f2e-4f13-902b-bd52964275a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:55.997669 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:55.997648 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6" pod "326136e5-1f2e-4f13-902b-bd52964275a2" (UID: "326136e5-1f2e-4f13-902b-bd52964275a2"). InnerVolumeSpecName "kube-api-access-qh9t6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:56.000468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.000439 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util" pod "326136e5-1f2e-4f13-902b-bd52964275a2" (UID: "326136e5-1f2e-4f13-902b-bd52964275a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:56.097105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097030 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\" (UniqueName: \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\") pod \"2e257f88-8761-4de8-a44b-af3039a98a28\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " Nov 04 05:31:56.097105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097102 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\") pod \"2e257f88-8761-4de8-a44b-af3039a98a28\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " Nov 04 05:31:56.097346 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097134 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\") pod \"2e257f88-8761-4de8-a44b-af3039a98a28\" (UID: \"2e257f88-8761-4de8-a44b-af3039a98a28\") " Nov 04 05:31:56.097413 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097373 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.097472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097419 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh9t6\" (UniqueName: \"kubernetes.io/projected/326136e5-1f2e-4f13-902b-bd52964275a2-kube-api-access-qh9t6\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.097472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097436 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/326136e5-1f2e-4f13-902b-bd52964275a2-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.097649 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.097625 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle" pod "2e257f88-8761-4de8-a44b-af3039a98a28" (UID: "2e257f88-8761-4de8-a44b-af3039a98a28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:56.099071 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.099050 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb" pod "2e257f88-8761-4de8-a44b-af3039a98a28" (UID: "2e257f88-8761-4de8-a44b-af3039a98a28"). InnerVolumeSpecName "kube-api-access-svxxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:56.102842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.102817 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util" pod "2e257f88-8761-4de8-a44b-af3039a98a28" (UID: "2e257f88-8761-4de8-a44b-af3039a98a28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:56.198353 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.198324 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svxxb\" (UniqueName: \"kubernetes.io/projected/2e257f88-8761-4de8-a44b-af3039a98a28-kube-api-access-svxxb\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.198353 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.198351 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.198512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.198365 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e257f88-8761-4de8-a44b-af3039a98a28-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:56.760673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.760642 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" Nov 04 05:31:56.761069 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.760641 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445d82z" event={"ID":"326136e5-1f2e-4f13-902b-bd52964275a2","Type":"ContainerDied","Data":"e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24"} Nov 04 05:31:56.761069 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.760759 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e704a792b40d3fd349b44e800513d12e299870f0c92f4adaeaeed2738dcf1d24" Nov 04 05:31:56.762935 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.762903 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" event={"ID":"2e257f88-8761-4de8-a44b-af3039a98a28","Type":"ContainerDied","Data":"c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb"} Nov 04 05:31:56.763038 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.762946 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4f2165dbbf0a404e603c8badbf055251e3d230f83300423c98ed8a869621ebb" Nov 04 05:31:56.763530 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.763516 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7smdt8" Nov 04 05:31:56.915790 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.915757 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:56.969452 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.969429 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:56.972077 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.972058 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:56.975203 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:56.975186 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:57.004755 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004733 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\") pod \"3797da46-4333-454b-bbdb-73eabb06a68a\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " Nov 04 05:31:57.004852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004800 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\") pod \"2120dba0-cdd4-4b0d-922d-e29dffda487c\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " Nov 04 05:31:57.004852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004817 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\") pod \"c00afd8a-366f-468a-9868-4e57b0f02fb5\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " Nov 04 05:31:57.004852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004846 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\" (UniqueName: \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\") pod \"3797da46-4333-454b-bbdb-73eabb06a68a\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " Nov 04 05:31:57.005014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004871 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\") pod \"2120dba0-cdd4-4b0d-922d-e29dffda487c\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " Nov 04 05:31:57.005014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004913 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\" (UniqueName: \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\") pod \"2120dba0-cdd4-4b0d-922d-e29dffda487c\" (UID: \"2120dba0-cdd4-4b0d-922d-e29dffda487c\") " Nov 04 05:31:57.005014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.004959 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\") pod \"3797da46-4333-454b-bbdb-73eabb06a68a\" (UID: \"3797da46-4333-454b-bbdb-73eabb06a68a\") " Nov 04 05:31:57.005014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.005000 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\" (UniqueName: \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\") pod \"5bb6320d-60c5-47aa-bc94-dd734b914c32\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " Nov 04 05:31:57.005217 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.005029 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\") pod \"c00afd8a-366f-468a-9868-4e57b0f02fb5\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " Nov 04 05:31:57.005827 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.005644 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle" pod "2120dba0-cdd4-4b0d-922d-e29dffda487c" (UID: "2120dba0-cdd4-4b0d-922d-e29dffda487c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.005827 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.005716 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle" pod "3797da46-4333-454b-bbdb-73eabb06a68a" (UID: "3797da46-4333-454b-bbdb-73eabb06a68a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.005827 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.005784 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle" pod "c00afd8a-366f-468a-9868-4e57b0f02fb5" (UID: "c00afd8a-366f-468a-9868-4e57b0f02fb5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.007512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.007481 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst" pod "2120dba0-cdd4-4b0d-922d-e29dffda487c" (UID: "2120dba0-cdd4-4b0d-922d-e29dffda487c"). InnerVolumeSpecName "kube-api-access-rqwst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:57.007622 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.007488 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz" pod "3797da46-4333-454b-bbdb-73eabb06a68a" (UID: "3797da46-4333-454b-bbdb-73eabb06a68a"). InnerVolumeSpecName "kube-api-access-m7gsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:57.007797 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.007778 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4" pod "5bb6320d-60c5-47aa-bc94-dd734b914c32" (UID: "5bb6320d-60c5-47aa-bc94-dd734b914c32"). InnerVolumeSpecName "kube-api-access-x6fz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:57.010948 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.010925 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util" pod "c00afd8a-366f-468a-9868-4e57b0f02fb5" (UID: "c00afd8a-366f-468a-9868-4e57b0f02fb5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.011100 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.011076 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util" pod "3797da46-4333-454b-bbdb-73eabb06a68a" (UID: "3797da46-4333-454b-bbdb-73eabb06a68a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.011791 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.011768 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util" pod "2120dba0-cdd4-4b0d-922d-e29dffda487c" (UID: "2120dba0-cdd4-4b0d-922d-e29dffda487c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.106152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106081 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\" (UniqueName: \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\") pod \"c00afd8a-366f-468a-9868-4e57b0f02fb5\" (UID: \"c00afd8a-366f-468a-9868-4e57b0f02fb5\") " Nov 04 05:31:57.106152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106116 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\") pod \"5bb6320d-60c5-47aa-bc94-dd734b914c32\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " Nov 04 05:31:57.106152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106137 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\") pod \"5bb6320d-60c5-47aa-bc94-dd734b914c32\" (UID: \"5bb6320d-60c5-47aa-bc94-dd734b914c32\") " Nov 04 05:31:57.106364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106315 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106333 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6fz4\" (UniqueName: \"kubernetes.io/projected/5bb6320d-60c5-47aa-bc94-dd734b914c32-kube-api-access-x6fz4\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106348 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106362 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3797da46-4333-454b-bbdb-73eabb06a68a-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106377 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106388 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00afd8a-366f-468a-9868-4e57b0f02fb5-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106400 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7gsz\" (UniqueName: \"kubernetes.io/projected/3797da46-4333-454b-bbdb-73eabb06a68a-kube-api-access-m7gsz\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106413 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2120dba0-cdd4-4b0d-922d-e29dffda487c-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106428 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqwst\" (UniqueName: \"kubernetes.io/projected/2120dba0-cdd4-4b0d-922d-e29dffda487c-kube-api-access-rqwst\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.106631 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.106537 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle" pod "5bb6320d-60c5-47aa-bc94-dd734b914c32" (UID: "5bb6320d-60c5-47aa-bc94-dd734b914c32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.107997 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.107974 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw" pod "c00afd8a-366f-468a-9868-4e57b0f02fb5" (UID: "c00afd8a-366f-468a-9868-4e57b0f02fb5"). InnerVolumeSpecName "kube-api-access-dhvlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:31:57.111732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.111710 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util" pod "5bb6320d-60c5-47aa-bc94-dd734b914c32" (UID: "5bb6320d-60c5-47aa-bc94-dd734b914c32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:31:57.206782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.206753 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhvlw\" (UniqueName: \"kubernetes.io/projected/c00afd8a-366f-468a-9868-4e57b0f02fb5-kube-api-access-dhvlw\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.206782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.206775 5104 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-util\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.206782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.206786 5104 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb6320d-60c5-47aa-bc94-dd734b914c32-bundle\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:31:57.412948 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.412865 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:57.768801 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.768771 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" Nov 04 05:31:57.769250 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.768770 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agj66q" event={"ID":"c00afd8a-366f-468a-9868-4e57b0f02fb5","Type":"ContainerDied","Data":"8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a"} Nov 04 05:31:57.769250 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.768921 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c7b85d165c0ff01ce9bee7944f48ba278a43b44a6cf84d2c9b2aad65b1eda7a" Nov 04 05:31:57.770790 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.770773 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" Nov 04 05:31:57.770919 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.770800 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6f2rdg" event={"ID":"2120dba0-cdd4-4b0d-922d-e29dffda487c","Type":"ContainerDied","Data":"3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056"} Nov 04 05:31:57.770919 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.770855 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d6d980b13ce4fd8ee8eb42f1baf70cbcdb6f003de14ff07d357b90b657dc056" Nov 04 05:31:57.772887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.772866 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" Nov 04 05:31:57.772983 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.772904 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbpbbx" event={"ID":"3797da46-4333-454b-bbdb-73eabb06a68a","Type":"ContainerDied","Data":"3377a28e00dbc6d945354fd4bf6329db4bd3c62414a68355efcf7c60abdfe951"} Nov 04 05:31:57.772983 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.772950 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3377a28e00dbc6d945354fd4bf6329db4bd3c62414a68355efcf7c60abdfe951" Nov 04 05:31:57.775068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.775051 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" Nov 04 05:31:57.775068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.775052 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644x6ck7" event={"ID":"5bb6320d-60c5-47aa-bc94-dd734b914c32","Type":"ContainerDied","Data":"6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b"} Nov 04 05:31:57.775247 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:57.775093 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b214e99b8738c0690ac75e2f058e6a6b5425587c5de7f2030ee9842d82b2d7b" Nov 04 05:31:58.159452 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159375 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps"] Nov 04 05:31:58.159682 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159669 5104 cpu_manager.go:414] "containerMap: removing container" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="extract" Nov 04 05:31:58.159721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159684 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="extract" Nov 04 05:31:58.159721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159696 5104 cpu_manager.go:414] "containerMap: removing container" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="util" Nov 04 05:31:58.159721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159701 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="util" Nov 04 05:31:58.159721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159709 5104 cpu_manager.go:414] "containerMap: removing container" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="pull" Nov 04 05:31:58.159721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159714 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159727 5104 cpu_manager.go:414] "containerMap: removing container" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159733 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159744 5104 cpu_manager.go:414] "containerMap: removing container" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159751 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159760 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159765 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159772 5104 cpu_manager.go:414] "containerMap: removing container" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159776 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159781 5104 cpu_manager.go:414] "containerMap: removing container" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159786 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159792 5104 cpu_manager.go:414] "containerMap: removing container" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159797 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159802 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159806 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159812 5104 cpu_manager.go:414] "containerMap: removing container" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159818 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159829 5104 cpu_manager.go:414] "containerMap: removing container" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159833 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159838 5104 cpu_manager.go:414] "containerMap: removing container" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159842 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159848 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159853 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="pull" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159858 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159862 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="util" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159866 5104 cpu_manager.go:414] "containerMap: removing container" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159871 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159876 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="extract" Nov 04 05:31:58.159869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159880 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159888 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159893 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159903 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159910 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159917 5104 cpu_manager.go:414] "containerMap: removing container" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159922 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159927 5104 cpu_manager.go:414] "containerMap: removing container" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159931 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159935 5104 cpu_manager.go:414] "containerMap: removing container" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159939 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159946 5104 cpu_manager.go:414] "containerMap: removing container" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159950 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159956 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159959 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159966 5104 cpu_manager.go:414] "containerMap: removing container" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159970 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="util" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159977 5104 cpu_manager.go:414] "containerMap: removing container" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159984 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159993 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.159997 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="pull" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160036 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb3d0c5e-667e-4b84-a0bf-82a894120f55" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160043 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="2120dba0-cdd4-4b0d-922d-e29dffda487c" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160047 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bb6320d-60c5-47aa-bc94-dd734b914c32" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160055 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="c00afd8a-366f-468a-9868-4e57b0f02fb5" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160065 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2fc1db1-9ad7-46a3-8761-eb072a877f05" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160075 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="326136e5-1f2e-4f13-902b-bd52964275a2" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160079 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="881ef3a8-1afb-4706-9586-ec682afb0cf4" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160084 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e257f88-8761-4de8-a44b-af3039a98a28" containerName="extract" Nov 04 05:31:58.160698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.160088 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="3797da46-4333-454b-bbdb-73eabb06a68a" containerName="extract" Nov 04 05:31:58.164404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.164389 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212008 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vmjb\" (UniqueName: \"kubernetes.io/projected/caa893e5-1670-49a6-8f6b-7d517cfebb9e-kube-api-access-4vmjb\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212049 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212137 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212245 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212270 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.212361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.212312 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.228093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.228070 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 05:31:58.230582 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.230565 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 05:31:58.230711 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.230627 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 05:31:58.230831 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.230790 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 05:31:58.231126 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.231113 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 05:31:58.231319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.231303 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-ssrcr\"" Nov 04 05:31:58.284656 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.284628 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps"] Nov 04 05:31:58.313438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313414 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313447 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313484 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vmjb\" (UniqueName: \"kubernetes.io/projected/caa893e5-1670-49a6-8f6b-7d517cfebb9e-kube-api-access-4vmjb\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313503 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313535 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313550 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.313949 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.313928 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.316122 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.316098 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.316378 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.316362 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.316423 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.316395 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caa893e5-1670-49a6-8f6b-7d517cfebb9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.316634 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.316615 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caa893e5-1670-49a6-8f6b-7d517cfebb9e-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.381859 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.381830 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vmjb\" (UniqueName: \"kubernetes.io/projected/caa893e5-1670-49a6-8f6b-7d517cfebb9e-kube-api-access-4vmjb\") pod \"opendatahub-operator-controller-manager-c98788444-dhhps\" (UID: \"caa893e5-1670-49a6-8f6b-7d517cfebb9e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.478263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.478232 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:31:58.946157 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:58.946125 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps"] Nov 04 05:31:58.947170 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:31:58.947147 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa893e5_1670_49a6_8f6b_7d517cfebb9e.slice/crio-b4a43c9841ba98cbd96189560ddb219c0a22d4954de200f586b72a03cdfa5bd4 WatchSource:0}: Error finding container b4a43c9841ba98cbd96189560ddb219c0a22d4954de200f586b72a03cdfa5bd4: Status 404 returned error can't find the container with id b4a43c9841ba98cbd96189560ddb219c0a22d4954de200f586b72a03cdfa5bd4 Nov 04 05:31:59.591021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:59.590994 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:31:59.784497 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:31:59.784467 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" event={"ID":"caa893e5-1670-49a6-8f6b-7d517cfebb9e","Type":"ContainerStarted","Data":"b4a43c9841ba98cbd96189560ddb219c0a22d4954de200f586b72a03cdfa5bd4"} Nov 04 05:32:00.493131 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:32:00.493094 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:32:00.817199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:00.817119 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:01.063352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:01.063310 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:01.425654 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:01.425603 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:01.638471 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:01.638423 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:02.700020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.699984 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz"] Nov 04 05:32:02.709014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.708992 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.751301 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.751272 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvc2\" (UniqueName: \"kubernetes.io/projected/7dad7e0c-46be-4b78-9792-bf42587dff3a-kube-api-access-vdvc2\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.751448 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.751430 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dad7e0c-46be-4b78-9792-bf42587dff3a-tmp\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.771908 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.771874 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz"] Nov 04 05:32:02.826873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.826833 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:32:02.827094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.827069 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-kt9w7\"" Nov 04 05:32:02.827175 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.827075 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Nov 04 05:32:02.852608 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.852581 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvc2\" (UniqueName: \"kubernetes.io/projected/7dad7e0c-46be-4b78-9792-bf42587dff3a-kube-api-access-vdvc2\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.852731 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.852694 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dad7e0c-46be-4b78-9792-bf42587dff3a-tmp\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.853028 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.853007 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dad7e0c-46be-4b78-9792-bf42587dff3a-tmp\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:02.924162 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:02.924130 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvc2\" (UniqueName: \"kubernetes.io/projected/7dad7e0c-46be-4b78-9792-bf42587dff3a-kube-api-access-vdvc2\") pod \"jobset-operator-674cd59dc5-hxfbz\" (UID: \"7dad7e0c-46be-4b78-9792-bf42587dff3a\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:03.037772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:03.037740 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" Nov 04 05:32:03.435338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:03.435263 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz"] Nov 04 05:32:03.438532 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:03.438501 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dad7e0c_46be_4b78_9792_bf42587dff3a.slice/crio-65c999e5c4b9227dfb43d51d10a47f9f62b5fc9edb0bc2fc9d5952d3ebec2fdb WatchSource:0}: Error finding container 65c999e5c4b9227dfb43d51d10a47f9f62b5fc9edb0bc2fc9d5952d3ebec2fdb: Status 404 returned error can't find the container with id 65c999e5c4b9227dfb43d51d10a47f9f62b5fc9edb0bc2fc9d5952d3ebec2fdb Nov 04 05:32:03.805212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:03.805180 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" event={"ID":"7dad7e0c-46be-4b78-9792-bf42587dff3a","Type":"ContainerStarted","Data":"65c999e5c4b9227dfb43d51d10a47f9f62b5fc9edb0bc2fc9d5952d3ebec2fdb"} Nov 04 05:32:04.811418 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:04.811375 5104 generic.go:356] "Generic (PLEG): container finished" podID="caa893e5-1670-49a6-8f6b-7d517cfebb9e" containerID="307c61e39a5a6f58c788b09bca28632718aef16db340a10f815c6def1acd6ed8" exitCode=0 Nov 04 05:32:04.811887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:04.811466 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" event={"ID":"caa893e5-1670-49a6-8f6b-7d517cfebb9e","Type":"ContainerDied","Data":"307c61e39a5a6f58c788b09bca28632718aef16db340a10f815c6def1acd6ed8"} Nov 04 05:32:05.818503 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:05.818469 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" event={"ID":"caa893e5-1670-49a6-8f6b-7d517cfebb9e","Type":"ContainerStarted","Data":"b1478fde88472b8bda8def7532ee3a019a3e954cb2825de8dcc0e107ae7e24a6"} Nov 04 05:32:05.818944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:05.818650 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:32:06.116904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:06.116848 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" podStartSLOduration=1.9259707019999999 podStartE2EDuration="8.116829877s" totalImagesPullingTime="5.743389628s" totalInitContainerRuntime="447.469547ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:06.059644917 +0000 UTC m=+263.624028217" watchObservedRunningTime="2026-11-04 05:32:06.116829877 +0000 UTC m=+263.681213181" Nov 04 05:32:07.431804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:07.431777 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:07.644676 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:07.644602 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:07.832732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:07.832696 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" event={"ID":"7dad7e0c-46be-4b78-9792-bf42587dff3a","Type":"ContainerStarted","Data":"3db391f9e876342fbb9414f3834de2bafe9a8039d8b39e28e60fc43248d823d8"} Nov 04 05:32:07.964584 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:07.964490 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hxfbz" podStartSLOduration=2.02449076 podStartE2EDuration="5.964476523s" totalImagesPullingTime="3.939985763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:07.963888501 +0000 UTC m=+265.528271801" watchObservedRunningTime="2026-11-04 05:32:07.964476523 +0000 UTC m=+265.528859821" Nov 04 05:32:10.422236 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:10.422190 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:10.701209 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:10.701133 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:13.457772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.457738 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5"] Nov 04 05:32:13.463911 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.463894 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:13.532584 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.532553 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj57k\" (UniqueName: \"kubernetes.io/projected/a4582504-452c-4a5d-a936-7218745f84b1-kube-api-access-sj57k\") pod \"dns-operator-controller-manager-8557cbdb84-djtz5\" (UID: \"a4582504-452c-4a5d-a936-7218745f84b1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:13.577586 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.577550 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Nov 04 05:32:13.577586 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.577550 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 05:32:13.577797 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.577565 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4rqcm\"" Nov 04 05:32:13.577797 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.577621 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 05:32:13.579263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.579241 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5"] Nov 04 05:32:13.633116 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.633083 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj57k\" (UniqueName: \"kubernetes.io/projected/a4582504-452c-4a5d-a936-7218745f84b1-kube-api-access-sj57k\") pod \"dns-operator-controller-manager-8557cbdb84-djtz5\" (UID: \"a4582504-452c-4a5d-a936-7218745f84b1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:13.699858 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.699831 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj57k\" (UniqueName: \"kubernetes.io/projected/a4582504-452c-4a5d-a936-7218745f84b1-kube-api-access-sj57k\") pod \"dns-operator-controller-manager-8557cbdb84-djtz5\" (UID: \"a4582504-452c-4a5d-a936-7218745f84b1\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:13.776786 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:13.776760 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:14.050018 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:14.049950 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:14.157269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:14.157236 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5"] Nov 04 05:32:14.160151 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:14.160121 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4582504_452c_4a5d_a936_7218745f84b1.slice/crio-7afa5b6269af7deaaccd85cfdd0811da1a4235f12bb4a61b2519947d1f55ebc9 WatchSource:0}: Error finding container 7afa5b6269af7deaaccd85cfdd0811da1a4235f12bb4a61b2519947d1f55ebc9: Status 404 returned error can't find the container with id 7afa5b6269af7deaaccd85cfdd0811da1a4235f12bb4a61b2519947d1f55ebc9 Nov 04 05:32:14.274357 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:14.274323 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:14.861141 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:14.861103 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" event={"ID":"a4582504-452c-4a5d-a936-7218745f84b1","Type":"ContainerStarted","Data":"7afa5b6269af7deaaccd85cfdd0811da1a4235f12bb4a61b2519947d1f55ebc9"} Nov 04 05:32:15.258702 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:15.258665 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:16.825789 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:16.825758 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-dhhps" Nov 04 05:32:17.660577 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.660543 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z"] Nov 04 05:32:17.670532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.670505 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.728775 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.728747 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zrzmp\"" Nov 04 05:32:17.738175 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.738146 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:32:17.738175 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.738150 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Nov 04 05:32:17.767552 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.767490 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.767704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.767557 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94zqx\" (UniqueName: \"kubernetes.io/projected/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-kube-api-access-94zqx\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.868969 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.868920 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94zqx\" (UniqueName: \"kubernetes.io/projected/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-kube-api-access-94zqx\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.869429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.869012 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.869486 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.869446 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.922492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.922447 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z"] Nov 04 05:32:17.936480 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.936451 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94zqx\" (UniqueName: \"kubernetes.io/projected/0a67beaa-bf5a-4f48-98d4-690a6c8576c9-kube-api-access-94zqx\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z\" (UID: \"0a67beaa-bf5a-4f48-98d4-690a6c8576c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:17.985521 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:17.985486 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" Nov 04 05:32:18.432307 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.432270 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z"] Nov 04 05:32:18.435467 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:18.435430 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a67beaa_bf5a_4f48_98d4_690a6c8576c9.slice/crio-0f2f2f95e1f71e7b079e78b6f67bab84a5cb68c7ddd82124ea94e4a5ea1b1688 WatchSource:0}: Error finding container 0f2f2f95e1f71e7b079e78b6f67bab84a5cb68c7ddd82124ea94e4a5ea1b1688: Status 404 returned error can't find the container with id 0f2f2f95e1f71e7b079e78b6f67bab84a5cb68c7ddd82124ea94e4a5ea1b1688 Nov 04 05:32:18.880490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.880444 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" event={"ID":"a4582504-452c-4a5d-a936-7218745f84b1","Type":"ContainerStarted","Data":"4dc6a4beed8fd4e0f9b04b3535fb7edbdcaa185e4ee0beee021c8d6e5aaa808f"} Nov 04 05:32:18.880957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.880519 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:18.880957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.880546 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq"] Nov 04 05:32:18.884683 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.884662 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" event={"ID":"0a67beaa-bf5a-4f48-98d4-690a6c8576c9","Type":"ContainerStarted","Data":"0f2f2f95e1f71e7b079e78b6f67bab84a5cb68c7ddd82124ea94e4a5ea1b1688"} Nov 04 05:32:18.884795 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.884737 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:18.978550 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:18.978514 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6z5p\" (UniqueName: \"kubernetes.io/projected/7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6-kube-api-access-d6z5p\") pod \"authorino-operator-7b7bb4bc77-mwkgq\" (UID: \"7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:19.002198 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.002168 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p6ltj\"" Nov 04 05:32:19.003606 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.003588 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq"] Nov 04 05:32:19.016683 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.016641 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" podStartSLOduration=1.932244173 podStartE2EDuration="6.016629668s" totalImagesPullingTime="4.084385495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:19.016148023 +0000 UTC m=+276.580531321" watchObservedRunningTime="2026-11-04 05:32:19.016629668 +0000 UTC m=+276.581012966" Nov 04 05:32:19.079703 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.079677 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6z5p\" (UniqueName: \"kubernetes.io/projected/7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6-kube-api-access-d6z5p\") pod \"authorino-operator-7b7bb4bc77-mwkgq\" (UID: \"7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:19.158704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.158637 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6z5p\" (UniqueName: \"kubernetes.io/projected/7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6-kube-api-access-d6z5p\") pod \"authorino-operator-7b7bb4bc77-mwkgq\" (UID: \"7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:19.197493 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.197468 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:19.380787 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.380756 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:19.582896 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.582735 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq"] Nov 04 05:32:19.585474 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:19.585443 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bcc7f7a_bdf3_48b8_9cbd_30da2838c3f6.slice/crio-c172f4dbdaaea22ff9a06ff8b9f6e93777a6c017e6e4367abb05c888cb61f95a WatchSource:0}: Error finding container c172f4dbdaaea22ff9a06ff8b9f6e93777a6c017e6e4367abb05c888cb61f95a: Status 404 returned error can't find the container with id c172f4dbdaaea22ff9a06ff8b9f6e93777a6c017e6e4367abb05c888cb61f95a Nov 04 05:32:19.610614 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.610549 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:19.886870 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:19.886787 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" event={"ID":"7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6","Type":"ContainerStarted","Data":"c172f4dbdaaea22ff9a06ff8b9f6e93777a6c017e6e4367abb05c888cb61f95a"} Nov 04 05:32:20.194880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:20.194778 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:20.419917 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:20.419884 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:20.653746 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:20.653708 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:20.997307 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:20.997270 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:21.271187 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:21.271109 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:22.903652 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:22.903614 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" event={"ID":"0a67beaa-bf5a-4f48-98d4-690a6c8576c9","Type":"ContainerStarted","Data":"cf0134acf76cf396d5e339cda79a13013baddb32a3ed70a31e6297caecd8765c"} Nov 04 05:32:22.905393 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:22.905352 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" event={"ID":"7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6","Type":"ContainerStarted","Data":"53aff74e4667a9b31ccafaba05c71e037c5a20a8adbbbb53f410523de6e15aec"} Nov 04 05:32:22.905536 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:22.905506 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:23.038516 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:23.038433 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:23.150239 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:23.150173 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-w2t2z" podStartSLOduration=1.871665425 podStartE2EDuration="6.150159059s" totalImagesPullingTime="4.278493634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:23.092729524 +0000 UTC m=+280.657112823" watchObservedRunningTime="2026-11-04 05:32:23.150159059 +0000 UTC m=+280.714542358" Nov 04 05:32:23.239717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:23.239668 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" podStartSLOduration=2.336291738 podStartE2EDuration="5.239652845s" totalImagesPullingTime="2.903361107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:23.239121166 +0000 UTC m=+280.803504464" watchObservedRunningTime="2026-11-04 05:32:23.239652845 +0000 UTC m=+280.804036143" Nov 04 05:32:24.731363 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:24.731331 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:24.952630 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:24.952590 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:26.158507 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.158479 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:26.325667 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.325633 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22"] Nov 04 05:32:26.329048 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.329031 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.388448 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.388417 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22"] Nov 04 05:32:26.395207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.395184 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:26.439985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.439924 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtc65\" (UniqueName: \"kubernetes.io/projected/67126a5b-679c-4a72-8803-d818835b6869-kube-api-access-gtc65\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.439985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.439968 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67126a5b-679c-4a72-8803-d818835b6869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.442898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.442874 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x72cp\"" Nov 04 05:32:26.541276 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.541245 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtc65\" (UniqueName: \"kubernetes.io/projected/67126a5b-679c-4a72-8803-d818835b6869-kube-api-access-gtc65\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.541450 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.541295 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67126a5b-679c-4a72-8803-d818835b6869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.541708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.541686 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67126a5b-679c-4a72-8803-d818835b6869-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.608841 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.608811 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtc65\" (UniqueName: \"kubernetes.io/projected/67126a5b-679c-4a72-8803-d818835b6869-kube-api-access-gtc65\") pod \"kuadrant-operator-controller-manager-678d87766f-b7r22\" (UID: \"67126a5b-679c-4a72-8803-d818835b6869\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.641597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.641573 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:26.927531 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:26.927493 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:27.143911 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:27.143884 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22"] Nov 04 05:32:27.145983 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:27.145951 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67126a5b_679c_4a72_8803_d818835b6869.slice/crio-4f044bd6313125224ed9583b4a1f2fa2475917b5cc8fbd4d300e9a9edfe85655 WatchSource:0}: Error finding container 4f044bd6313125224ed9583b4a1f2fa2475917b5cc8fbd4d300e9a9edfe85655: Status 404 returned error can't find the container with id 4f044bd6313125224ed9583b4a1f2fa2475917b5cc8fbd4d300e9a9edfe85655 Nov 04 05:32:27.156396 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:27.156363 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:27.582685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:27.582644 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:27.928689 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:27.928609 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" event={"ID":"67126a5b-679c-4a72-8803-d818835b6869","Type":"ContainerStarted","Data":"4f044bd6313125224ed9583b4a1f2fa2475917b5cc8fbd4d300e9a9edfe85655"} Nov 04 05:32:29.021891 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.021864 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:29.245355 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.245313 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:29.670527 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.670498 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m"] Nov 04 05:32:29.673667 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.673648 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:29.770192 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.770153 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzc2d\" (UniqueName: \"kubernetes.io/projected/48ae5a00-e13e-4f71-ab93-07b8d6fa5716-kube-api-access-qzc2d\") pod \"limitador-operator-controller-manager-5778bb4449-mlw5m\" (UID: \"48ae5a00-e13e-4f71-ab93-07b8d6fa5716\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:29.788829 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.788797 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9blkh\"" Nov 04 05:32:29.790396 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.790377 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m"] Nov 04 05:32:29.870641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.870608 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzc2d\" (UniqueName: \"kubernetes.io/projected/48ae5a00-e13e-4f71-ab93-07b8d6fa5716-kube-api-access-qzc2d\") pod \"limitador-operator-controller-manager-5778bb4449-mlw5m\" (UID: \"48ae5a00-e13e-4f71-ab93-07b8d6fa5716\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:29.889312 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.889289 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-djtz5" Nov 04 05:32:29.941839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.941769 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzc2d\" (UniqueName: \"kubernetes.io/projected/48ae5a00-e13e-4f71-ab93-07b8d6fa5716-kube-api-access-qzc2d\") pod \"limitador-operator-controller-manager-5778bb4449-mlw5m\" (UID: \"48ae5a00-e13e-4f71-ab93-07b8d6fa5716\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:29.986265 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:29.986240 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:30.377919 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:30.377881 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m"] Nov 04 05:32:30.383191 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:30.383159 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ae5a00_e13e_4f71_ab93_07b8d6fa5716.slice/crio-ff6e6457ea17c3702970b726b9fde96d3a805a4d5846e6de3995e77835a9d8a5 WatchSource:0}: Error finding container ff6e6457ea17c3702970b726b9fde96d3a805a4d5846e6de3995e77835a9d8a5: Status 404 returned error can't find the container with id ff6e6457ea17c3702970b726b9fde96d3a805a4d5846e6de3995e77835a9d8a5 Nov 04 05:32:30.493442 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:32:30.493401 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:32:30.943279 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:30.943245 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" event={"ID":"48ae5a00-e13e-4f71-ab93-07b8d6fa5716","Type":"ContainerStarted","Data":"ff6e6457ea17c3702970b726b9fde96d3a805a4d5846e6de3995e77835a9d8a5"} Nov 04 05:32:31.854477 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:31.854445 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:32.084700 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:32.084616 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:32.294958 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:32.294924 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:32.671744 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:32.671674 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:33.912642 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:33.912605 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-mwkgq" Nov 04 05:32:34.087306 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.087240 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:34.470838 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.470810 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:34.662802 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.662770 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh"] Nov 04 05:32:34.666172 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.666154 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.693086 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.693060 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:34.735918 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.735842 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-knb6t\"" Nov 04 05:32:34.738129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.738104 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh"] Nov 04 05:32:34.814023 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.813976 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814034 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814164 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814256 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814303 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814373 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frmd4\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-kube-api-access-frmd4\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.814734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.814407 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.818261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.815427 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.818261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.815507 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916057 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916018 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916079 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916132 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916163 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916192 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916246 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916284 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916316 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frmd4\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-kube-api-access-frmd4\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.916596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.916347 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.917137 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.917110 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.917481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.917460 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.918326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.918303 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.918600 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.918564 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.919436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.919374 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.921162 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.920487 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.921775 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.921752 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.961462 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.961427 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:34.967667 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.967635 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" event={"ID":"67126a5b-679c-4a72-8803-d818835b6869","Type":"ContainerStarted","Data":"ac690e803c267c23525f9c6ab9d108ba2874ca775828e49e3fe89f3bd75e9451"} Nov 04 05:32:34.967800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.967768 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:34.989703 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.989644 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:34.989875 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:34.989853 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frmd4\" (UniqueName: \"kubernetes.io/projected/0eef5bbb-45d9-4ccd-b519-c8a2009b1704-kube-api-access-frmd4\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485xfwsh\" (UID: \"0eef5bbb-45d9-4ccd-b519-c8a2009b1704\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:35.112712 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.112647 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" podStartSLOduration=2.2550379879999998 podStartE2EDuration="9.112627277s" totalImagesPullingTime="6.857589289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:35.110665301 +0000 UTC m=+292.675048600" watchObservedRunningTime="2026-11-04 05:32:35.112627277 +0000 UTC m=+292.677010577" Nov 04 05:32:35.267099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.267026 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:35.279033 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.279007 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:35.475598 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.475507 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:35.736716 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.736686 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh"] Nov 04 05:32:35.740253 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:35.739516 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eef5bbb_45d9_4ccd_b519_c8a2009b1704.slice/crio-b872927be309e1031dbe03e73b577f154cdc8616ac5aaa921de86a4664b61e7d WatchSource:0}: Error finding container b872927be309e1031dbe03e73b577f154cdc8616ac5aaa921de86a4664b61e7d: Status 404 returned error can't find the container with id b872927be309e1031dbe03e73b577f154cdc8616ac5aaa921de86a4664b61e7d Nov 04 05:32:35.742267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.742211 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:32:35.742348 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.742313 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:32:35.742348 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.742341 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:32:35.974390 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.974356 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" event={"ID":"48ae5a00-e13e-4f71-ab93-07b8d6fa5716","Type":"ContainerStarted","Data":"a0430ef793a01554b05a36de5ff9ac48661c666c7801f6ff1952da7410cd21a8"} Nov 04 05:32:35.974821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.974424 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:35.976092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.976071 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" event={"ID":"0eef5bbb-45d9-4ccd-b519-c8a2009b1704","Type":"ContainerStarted","Data":"afd7c1b44047738ebd111e665f8e9d7884dd902f33f2711fcfc80b358b34bae0"} Nov 04 05:32:35.976193 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:35.976114 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" event={"ID":"0eef5bbb-45d9-4ccd-b519-c8a2009b1704","Type":"ContainerStarted","Data":"b872927be309e1031dbe03e73b577f154cdc8616ac5aaa921de86a4664b61e7d"} Nov 04 05:32:36.105550 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.105499 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" podStartSLOduration=1.83535363 podStartE2EDuration="7.105484008s" totalImagesPullingTime="5.270130378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:36.104053448 +0000 UTC m=+293.668436749" watchObservedRunningTime="2026-11-04 05:32:36.105484008 +0000 UTC m=+293.669867306" Nov 04 05:32:36.243782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.239637 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" podStartSLOduration=2.239617246 podStartE2EDuration="2.239617246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:36.237598164 +0000 UTC m=+293.801981462" watchObservedRunningTime="2026-11-04 05:32:36.239617246 +0000 UTC m=+293.804000545" Nov 04 05:32:36.279737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.279702 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:36.284621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.284596 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:36.981036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.980999 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:36.982013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:36.981991 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485xfwsh" Nov 04 05:32:37.802357 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:37.802310 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:38.009999 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:38.009968 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:38.805444 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:38.805407 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:39.047117 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:39.047083 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:41.453641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.453588 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:41.466686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.466655 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv"] Nov 04 05:32:41.471574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.471553 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.532135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.532102 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Nov 04 05:32:41.571661 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.571630 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnngk\" (UniqueName: \"kubernetes.io/projected/f57d437b-159e-4b38-b09b-f3d092f3cf26-kube-api-access-vnngk\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.571820 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.571687 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57d437b-159e-4b38-b09b-f3d092f3cf26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.571820 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.571710 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f57d437b-159e-4b38-b09b-f3d092f3cf26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.588925 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.588892 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6rl8q\"" Nov 04 05:32:41.588925 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.588891 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Nov 04 05:32:41.590559 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.590539 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv"] Nov 04 05:32:41.660452 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.660423 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:41.672620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.672593 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnngk\" (UniqueName: \"kubernetes.io/projected/f57d437b-159e-4b38-b09b-f3d092f3cf26-kube-api-access-vnngk\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.672739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.672640 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57d437b-159e-4b38-b09b-f3d092f3cf26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.672739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.672656 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f57d437b-159e-4b38-b09b-f3d092f3cf26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.673285 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.673261 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f57d437b-159e-4b38-b09b-f3d092f3cf26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.675127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.675101 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57d437b-159e-4b38-b09b-f3d092f3cf26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.740241 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.740197 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnngk\" (UniqueName: \"kubernetes.io/projected/f57d437b-159e-4b38-b09b-f3d092f3cf26-kube-api-access-vnngk\") pod \"kuadrant-console-plugin-66c4fb575d-vhlkv\" (UID: \"f57d437b-159e-4b38-b09b-f3d092f3cf26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.785746 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.785709 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" Nov 04 05:32:41.868423 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:41.868391 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:42.076272 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:42.076159 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:42.167261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:42.167050 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv"] Nov 04 05:32:42.170293 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:42.170254 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf57d437b_159e_4b38_b09b_f3d092f3cf26.slice/crio-3a7469e469bdd6b5442ec19cdd3bb83b95d0618d599e06155b5f323acd119176 WatchSource:0}: Error finding container 3a7469e469bdd6b5442ec19cdd3bb83b95d0618d599e06155b5f323acd119176: Status 404 returned error can't find the container with id 3a7469e469bdd6b5442ec19cdd3bb83b95d0618d599e06155b5f323acd119176 Nov 04 05:32:42.357313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:42.357206 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:43.010558 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.010521 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" event={"ID":"f57d437b-159e-4b38-b09b-f3d092f3cf26","Type":"ContainerStarted","Data":"3a7469e469bdd6b5442ec19cdd3bb83b95d0618d599e06155b5f323acd119176"} Nov 04 05:32:43.122126 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.122097 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:43.126173 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.126145 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:43.186381 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.186355 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hdh2w\"" Nov 04 05:32:43.187944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.187918 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:43.243174 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.243148 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:43.284369 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.284264 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzdpm\" (UniqueName: \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\") pod \"authorino-57f7f8659f-x4x84\" (UID: \"45bee80f-b63e-418e-8448-1c93a3ff5d9e\") " pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:43.385753 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.385721 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzdpm\" (UniqueName: \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\") pod \"authorino-57f7f8659f-x4x84\" (UID: \"45bee80f-b63e-418e-8448-1c93a3ff5d9e\") " pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:43.448121 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.448089 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:43.453176 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.453155 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzdpm\" (UniqueName: \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\") pod \"authorino-57f7f8659f-x4x84\" (UID: \"45bee80f-b63e-418e-8448-1c93a3ff5d9e\") " pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:43.650718 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.650640 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:43.737518 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:43.737484 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:44.118497 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:44.118468 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:44.121071 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:44.121046 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45bee80f_b63e_418e_8448_1c93a3ff5d9e.slice/crio-11616a8aa76b03c8aaefd72640334eaea19bded3e88eeb3d151b4f2087877983 WatchSource:0}: Error finding container 11616a8aa76b03c8aaefd72640334eaea19bded3e88eeb3d151b4f2087877983: Status 404 returned error can't find the container with id 11616a8aa76b03c8aaefd72640334eaea19bded3e88eeb3d151b4f2087877983 Nov 04 05:32:44.122455 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:44.122438 5104 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:32:45.024128 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:45.024089 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x4x84" event={"ID":"45bee80f-b63e-418e-8448-1c93a3ff5d9e","Type":"ContainerStarted","Data":"11616a8aa76b03c8aaefd72640334eaea19bded3e88eeb3d151b4f2087877983"} Nov 04 05:32:45.978534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:45.978504 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b7r22" Nov 04 05:32:45.988347 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:45.988314 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:32:45.991942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:45.991920 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:32:46.096484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.096450 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:32:46.110769 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.110739 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8stts\" (UniqueName: \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\") pod \"authorino-56c6c9ddd5-t5wpr\" (UID: \"8ce86675-48f4-4da4-b55f-604994e1ee3a\") " pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:32:46.128721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.128689 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:46.211638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.211600 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8stts\" (UniqueName: \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\") pod \"authorino-56c6c9ddd5-t5wpr\" (UID: \"8ce86675-48f4-4da4-b55f-604994e1ee3a\") " pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:32:46.279464 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.279383 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8stts\" (UniqueName: \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\") pod \"authorino-56c6c9ddd5-t5wpr\" (UID: \"8ce86675-48f4-4da4-b55f-604994e1ee3a\") " pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:32:46.305087 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.305047 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:32:46.388374 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.388336 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:46.712588 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.712559 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:32:46.714386 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:32:46.714354 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ce86675_48f4_4da4_b55f_604994e1ee3a.slice/crio-6b8e4582e38a01a1a594f7291381657c7bc08fc7b385d25962fee12e9f59086f WatchSource:0}: Error finding container 6b8e4582e38a01a1a594f7291381657c7bc08fc7b385d25962fee12e9f59086f: Status 404 returned error can't find the container with id 6b8e4582e38a01a1a594f7291381657c7bc08fc7b385d25962fee12e9f59086f Nov 04 05:32:46.983308 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:46.983274 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:32:47.037971 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:47.037937 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" event={"ID":"8ce86675-48f4-4da4-b55f-604994e1ee3a","Type":"ContainerStarted","Data":"6b8e4582e38a01a1a594f7291381657c7bc08fc7b385d25962fee12e9f59086f"} Nov 04 05:32:48.044771 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:48.044728 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x4x84" event={"ID":"45bee80f-b63e-418e-8448-1c93a3ff5d9e","Type":"ContainerStarted","Data":"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587"} Nov 04 05:32:48.173780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:48.173689 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-x4x84" podStartSLOduration=1.485376089 podStartE2EDuration="5.173671178s" totalImagesPullingTime="3.688295089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:48.172281949 +0000 UTC m=+305.736665247" watchObservedRunningTime="2026-11-04 05:32:48.173671178 +0000 UTC m=+305.738054476" Nov 04 05:32:49.021399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:49.021367 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:49.050669 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:49.050629 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" event={"ID":"8ce86675-48f4-4da4-b55f-604994e1ee3a","Type":"ContainerStarted","Data":"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a"} Nov 04 05:32:49.177131 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:49.177083 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" podStartSLOduration=2.288246418 podStartE2EDuration="4.177070493s" totalImagesPullingTime="1.888824075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:49.176621823 +0000 UTC m=+306.741005126" watchObservedRunningTime="2026-11-04 05:32:49.177070493 +0000 UTC m=+306.741453791" Nov 04 05:32:49.226550 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:49.226513 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:49.234795 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:49.234763 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:50.056791 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.056748 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" event={"ID":"f57d437b-159e-4b38-b09b-f3d092f3cf26","Type":"ContainerStarted","Data":"2bb2012d89ff83882869ce395f5a071b1c6456a30eb993c1bd7cbd3fb5893e3a"} Nov 04 05:32:50.057397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.056877 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-x4x84" podUID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" containerName="authorino" containerID="cri-o://b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587" gracePeriod=30 Nov 04 05:32:50.125823 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.125785 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:50.184705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.184650 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-vhlkv" podStartSLOduration=2.229362874 podStartE2EDuration="9.184634322s" totalImagesPullingTime="6.955271448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:50.184389634 +0000 UTC m=+307.748772933" watchObservedRunningTime="2026-11-04 05:32:50.184634322 +0000 UTC m=+307.749017622" Nov 04 05:32:50.315755 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.315702 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:50.337873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.337842 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:50.453858 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.453826 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\" (UniqueName: \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\") pod \"45bee80f-b63e-418e-8448-1c93a3ff5d9e\" (UID: \"45bee80f-b63e-418e-8448-1c93a3ff5d9e\") " Nov 04 05:32:50.455937 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.455912 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm" pod "45bee80f-b63e-418e-8448-1c93a3ff5d9e" (UID: "45bee80f-b63e-418e-8448-1c93a3ff5d9e"). InnerVolumeSpecName "kube-api-access-xzdpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:32:50.554717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:50.554685 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzdpm\" (UniqueName: \"kubernetes.io/projected/45bee80f-b63e-418e-8448-1c93a3ff5d9e-kube-api-access-xzdpm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:32:51.062724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.062691 5104 generic.go:356] "Generic (PLEG): container finished" podID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" containerID="b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587" exitCode=0 Nov 04 05:32:51.063315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.062739 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x4x84" Nov 04 05:32:51.063315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.062769 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x4x84" event={"ID":"45bee80f-b63e-418e-8448-1c93a3ff5d9e","Type":"ContainerDied","Data":"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587"} Nov 04 05:32:51.063315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.062812 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x4x84" event={"ID":"45bee80f-b63e-418e-8448-1c93a3ff5d9e","Type":"ContainerDied","Data":"11616a8aa76b03c8aaefd72640334eaea19bded3e88eeb3d151b4f2087877983"} Nov 04 05:32:51.063315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.062830 5104 scope.go:122] "RemoveContainer" containerID="b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587" Nov 04 05:32:51.071185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.071167 5104 scope.go:122] "RemoveContainer" containerID="b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587" Nov 04 05:32:51.071462 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:32:51.071442 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587\": container with ID starting with b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587 not found: ID does not exist" containerID="b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587" Nov 04 05:32:51.071574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.071469 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587"} err="failed to get container status \"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587\": rpc error: code = NotFound desc = could not find container \"b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587\": container with ID starting with b4165e49c0a015003df06d4bdceaadcb490551c4eaf62323b5a8ebe1b9cf0587 not found: ID does not exist" Nov 04 05:32:51.248175 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.248145 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:51.249618 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:51.249596 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x4x84"] Nov 04 05:32:52.549686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:52.549655 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" path="/var/lib/kubelet/pods/45bee80f-b63e-418e-8448-1c93a3ff5d9e/volumes" Nov 04 05:32:52.638652 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:52.638621 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:52.849301 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:52.849192 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:57.262539 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:57.262510 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:57.490187 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:57.490151 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:59.260523 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:59.260470 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:59.468794 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:59.468760 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:32:59.675545 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:32:59.675463 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:00.490660 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:33:00.490625 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:33:01.899986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:01.899954 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:02.394412 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:02.394381 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:02.601369 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:02.601323 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:03.598074 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:03.598035 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:03.799533 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:03.799488 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:04.803963 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:04.803924 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:05.025669 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.025619 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:05.736603 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.736571 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4"] Nov 04 05:33:05.736920 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.736905 5104 cpu_manager.go:414] "containerMap: removing container" podUID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" containerName="authorino" Nov 04 05:33:05.736974 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.736921 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" containerName="authorino" Nov 04 05:33:05.737009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.736990 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="45bee80f-b63e-418e-8448-1c93a3ff5d9e" containerName="authorino" Nov 04 05:33:05.741646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.741612 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.774741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.774710 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1efd7bb-616c-43c9-bd7c-aad1115a978c-data\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.774885 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.774773 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl8v2\" (UniqueName: \"kubernetes.io/projected/e1efd7bb-616c-43c9-bd7c-aad1115a978c-kube-api-access-xl8v2\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.802501 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.802466 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:33:05.802644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.802471 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:33:05.859079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.859041 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 04 05:33:05.859079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.859053 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fd6h2\"" Nov 04 05:33:05.860812 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.860791 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4"] Nov 04 05:33:05.875889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.875861 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1efd7bb-616c-43c9-bd7c-aad1115a978c-data\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.876043 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.875914 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl8v2\" (UniqueName: \"kubernetes.io/projected/e1efd7bb-616c-43c9-bd7c-aad1115a978c-kube-api-access-xl8v2\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.876428 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.876401 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1efd7bb-616c-43c9-bd7c-aad1115a978c-data\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:05.894392 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.894365 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:05.943614 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:05.943583 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl8v2\" (UniqueName: \"kubernetes.io/projected/e1efd7bb-616c-43c9-bd7c-aad1115a978c-kube-api-access-xl8v2\") pod \"postgres-7cdbf6dc44-bc2r4\" (UID: \"e1efd7bb-616c-43c9-bd7c-aad1115a978c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:06.053559 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:06.053476 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:06.441425 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:06.441165 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4"] Nov 04 05:33:06.444050 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:33:06.444005 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1efd7bb_616c_43c9_bd7c_aad1115a978c.slice/crio-0dda3e79fe2f9f954e31fcaf37859426eb4357abb08d5917ae9a678c5dc535d5 WatchSource:0}: Error finding container 0dda3e79fe2f9f954e31fcaf37859426eb4357abb08d5917ae9a678c5dc535d5: Status 404 returned error can't find the container with id 0dda3e79fe2f9f954e31fcaf37859426eb4357abb08d5917ae9a678c5dc535d5 Nov 04 05:33:07.144355 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:07.144321 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" event={"ID":"e1efd7bb-616c-43c9-bd7c-aad1115a978c","Type":"ContainerStarted","Data":"0dda3e79fe2f9f954e31fcaf37859426eb4357abb08d5917ae9a678c5dc535d5"} Nov 04 05:33:10.409688 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:10.409645 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:10.653059 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:10.653020 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:13.012476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:13.012433 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:13.219423 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:13.219395 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:14.185532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:14.185497 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" event={"ID":"e1efd7bb-616c-43c9-bd7c-aad1115a978c","Type":"ContainerStarted","Data":"0b937611794d95815e68069520cebbc1bec5b6ab1b1dd1b19e6ecefcb9df2fa2"} Nov 04 05:33:14.185905 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:14.185573 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:14.314594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:14.314499 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" podStartSLOduration=1.782741797 podStartE2EDuration="9.314486392s" totalImagesPullingTime="7.531744595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:14.313262377 +0000 UTC m=+331.877645675" watchObservedRunningTime="2026-11-04 05:33:14.314486392 +0000 UTC m=+331.878869691" Nov 04 05:33:18.417666 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:18.417623 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:18.622014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:18.621975 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:18.827123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:18.827077 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:19.027904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:19.027868 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:20.219343 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:20.219315 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-bc2r4" Nov 04 05:33:23.074511 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:23.074476 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:23.280015 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:23.279981 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:27.780038 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.780004 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:27.783520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.783496 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.784026 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.783989 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:27.852023 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.851994 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.852186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.852038 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.852186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.852141 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsxg4\" (UniqueName: \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.899030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.898997 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Nov 04 05:33:27.899030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.899017 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Nov 04 05:33:27.899030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.899006 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 05:33:27.899364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.899083 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dhmqr\"" Nov 04 05:33:27.899364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.899099 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 05:33:27.899364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.899111 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Nov 04 05:33:27.901069 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.901036 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:27.953290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.953255 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.953429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.953293 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.953429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.953320 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsxg4\" (UniqueName: \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.955755 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.955734 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.955813 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.955791 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:27.982377 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:27.982345 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:28.020421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.020395 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsxg4\" (UniqueName: \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\") pod \"maas-controller-8b95c9c66-4nj68\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:28.095416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.095352 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:28.175966 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.175933 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d77d66d96-nq9gq"] Nov 04 05:33:28.179359 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.179337 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.188501 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.188475 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:28.270386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.270349 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-metrics-certs\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.270554 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.270389 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7xtj\" (UniqueName: \"kubernetes.io/projected/c93362fa-d4a4-4786-b27a-c46db1b6084f-kube-api-access-l7xtj\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.270554 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.270533 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-webhook-cert\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.271082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.271058 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d77d66d96-nq9gq"] Nov 04 05:33:28.371438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.371359 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-metrics-certs\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.371438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.371397 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7xtj\" (UniqueName: \"kubernetes.io/projected/c93362fa-d4a4-4786-b27a-c46db1b6084f-kube-api-access-l7xtj\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.371655 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.371448 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-webhook-cert\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.373800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.373774 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-metrics-certs\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.373903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.373851 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93362fa-d4a4-4786-b27a-c46db1b6084f-webhook-cert\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.438892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.438864 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7xtj\" (UniqueName: \"kubernetes.io/projected/c93362fa-d4a4-4786-b27a-c46db1b6084f-kube-api-access-l7xtj\") pod \"maas-controller-7d77d66d96-nq9gq\" (UID: \"c93362fa-d4a4-4786-b27a-c46db1b6084f\") " pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.479835 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.479807 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:28.482330 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:33:28.482297 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e14d7d0_8419_4aaf_8c47_51df2542b11d.slice/crio-170198105c6a506ce2049c4b87bee9957fedbe475884221bab388c17c2fccb32 WatchSource:0}: Error finding container 170198105c6a506ce2049c4b87bee9957fedbe475884221bab388c17c2fccb32: Status 404 returned error can't find the container with id 170198105c6a506ce2049c4b87bee9957fedbe475884221bab388c17c2fccb32 Nov 04 05:33:28.492342 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.492320 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:28.878830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:28.878737 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d77d66d96-nq9gq"] Nov 04 05:33:28.881501 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:33:28.881477 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93362fa_d4a4_4786_b27a_c46db1b6084f.slice/crio-68540082c373ce421914a515b776026c638f8dd488a5890be0b30363bc11686f WatchSource:0}: Error finding container 68540082c373ce421914a515b776026c638f8dd488a5890be0b30363bc11686f: Status 404 returned error can't find the container with id 68540082c373ce421914a515b776026c638f8dd488a5890be0b30363bc11686f Nov 04 05:33:29.280199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:29.280166 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-4nj68" event={"ID":"2e14d7d0-8419-4aaf-8c47-51df2542b11d","Type":"ContainerStarted","Data":"170198105c6a506ce2049c4b87bee9957fedbe475884221bab388c17c2fccb32"} Nov 04 05:33:29.281347 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:29.281315 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" event={"ID":"c93362fa-d4a4-4786-b27a-c46db1b6084f","Type":"ContainerStarted","Data":"68540082c373ce421914a515b776026c638f8dd488a5890be0b30363bc11686f"} Nov 04 05:33:30.123536 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:30.123506 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:30.327108 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:30.327074 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:30.503731 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:33:30.503636 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:33:32.302099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:32.302062 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-4nj68" event={"ID":"2e14d7d0-8419-4aaf-8c47-51df2542b11d","Type":"ContainerStarted","Data":"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348"} Nov 04 05:33:32.302584 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:32.302202 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:32.434358 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:32.434311 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8b95c9c66-4nj68" podStartSLOduration=1.836850297 podStartE2EDuration="5.434297668s" totalImagesPullingTime="3.597447371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:32.432634677 +0000 UTC m=+349.997017988" watchObservedRunningTime="2026-11-04 05:33:32.434297668 +0000 UTC m=+349.998681008" Nov 04 05:33:32.650099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:32.650008 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:32.868736 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:32.868700 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:33.308250 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:33.308198 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" event={"ID":"c93362fa-d4a4-4786-b27a-c46db1b6084f","Type":"ContainerStarted","Data":"2fc74ec6ca8d6d199ddfea7abeaf1892dea3359f49b2095c4b36f0b979e0f37b"} Nov 04 05:33:33.308711 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:33.308285 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:33.439496 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:33.439448 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" podStartSLOduration=1.971850004 podStartE2EDuration="5.439436367s" totalImagesPullingTime="3.467586363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:33.439086913 +0000 UTC m=+351.003470223" watchObservedRunningTime="2026-11-04 05:33:33.439436367 +0000 UTC m=+351.003819665" Nov 04 05:33:35.578684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:35.578650 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:35.826807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:35.826769 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:43.311237 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:43.311189 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:44.315519 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.315491 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d77d66d96-nq9gq" Nov 04 05:33:44.506325 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.506288 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:44.506503 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.506478 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-8b95c9c66-4nj68" podUID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" containerName="manager" containerID="cri-o://8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348" gracePeriod=30 Nov 04 05:33:44.749963 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.749938 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:44.922012 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.921938 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\" (UniqueName: \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\") pod \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " Nov 04 05:33:44.922012 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.922001 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\") pod \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " Nov 04 05:33:44.922201 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.922111 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\") pod \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\" (UID: \"2e14d7d0-8419-4aaf-8c47-51df2542b11d\") " Nov 04 05:33:44.924442 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.924404 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert" pod "2e14d7d0-8419-4aaf-8c47-51df2542b11d" (UID: "2e14d7d0-8419-4aaf-8c47-51df2542b11d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:33:44.924561 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.924419 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs" pod "2e14d7d0-8419-4aaf-8c47-51df2542b11d" (UID: "2e14d7d0-8419-4aaf-8c47-51df2542b11d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:33:44.924561 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:44.924509 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4" pod "2e14d7d0-8419-4aaf-8c47-51df2542b11d" (UID: "2e14d7d0-8419-4aaf-8c47-51df2542b11d"). InnerVolumeSpecName "kube-api-access-hsxg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:33:45.023717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.023685 5104 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-webhook-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:45.023717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.023714 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsxg4\" (UniqueName: \"kubernetes.io/projected/2e14d7d0-8419-4aaf-8c47-51df2542b11d-kube-api-access-hsxg4\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:45.023717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.023725 5104 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e14d7d0-8419-4aaf-8c47-51df2542b11d-metrics-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:45.366665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.366628 5104 generic.go:356] "Generic (PLEG): container finished" podID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" containerID="8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348" exitCode=0 Nov 04 05:33:45.367080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.366672 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-4nj68" Nov 04 05:33:45.367080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.366717 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-4nj68" event={"ID":"2e14d7d0-8419-4aaf-8c47-51df2542b11d","Type":"ContainerDied","Data":"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348"} Nov 04 05:33:45.367080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.366772 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-4nj68" event={"ID":"2e14d7d0-8419-4aaf-8c47-51df2542b11d","Type":"ContainerDied","Data":"170198105c6a506ce2049c4b87bee9957fedbe475884221bab388c17c2fccb32"} Nov 04 05:33:45.367080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.366801 5104 scope.go:122] "RemoveContainer" containerID="8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348" Nov 04 05:33:45.376036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.376018 5104 scope.go:122] "RemoveContainer" containerID="8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348" Nov 04 05:33:45.376314 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:33:45.376293 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348\": container with ID starting with 8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348 not found: ID does not exist" containerID="8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348" Nov 04 05:33:45.376362 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.376324 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348"} err="failed to get container status \"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348\": rpc error: code = NotFound desc = could not find container \"8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348\": container with ID starting with 8c0cc45131eaac364bd2b8d3269100b16211ef819f885be4285a320c96be3348 not found: ID does not exist" Nov 04 05:33:45.551209 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.551173 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:45.607730 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:45.607700 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-4nj68"] Nov 04 05:33:46.548571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:46.548539 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" path="/var/lib/kubelet/pods/2e14d7d0-8419-4aaf-8c47-51df2542b11d/volumes" Nov 04 05:33:54.442194 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:54.441995 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:54.646083 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:54.646046 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:54.844702 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:54.844665 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:55.047255 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.047206 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:55.259450 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.259421 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:55.263001 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.262979 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-tn2gh"] Nov 04 05:33:55.263352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.263336 5104 cpu_manager.go:414] "containerMap: removing container" podUID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" containerName="manager" Nov 04 05:33:55.263418 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.263355 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" containerName="manager" Nov 04 05:33:55.263458 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.263432 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e14d7d0-8419-4aaf-8c47-51df2542b11d" containerName="manager" Nov 04 05:33:55.266495 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.266480 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.304290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.304260 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8pnk\" (UniqueName: \"kubernetes.io/projected/ac436778-eb3c-42c3-8187-fe3f0289957a-kube-api-access-l8pnk\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.304408 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.304318 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac436778-eb3c-42c3-8187-fe3f0289957a-cert\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.381867 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.381810 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Nov 04 05:33:55.381867 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.381810 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zbshb\"" Nov 04 05:33:55.383800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.383774 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-tn2gh"] Nov 04 05:33:55.385309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.385288 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:33:55.388571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.388556 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.404740 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.404713 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4mhr\" (UniqueName: \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.404832 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.404749 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac436778-eb3c-42c3-8187-fe3f0289957a-cert\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.404883 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.404863 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.404971 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.404958 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8pnk\" (UniqueName: \"kubernetes.io/projected/ac436778-eb3c-42c3-8187-fe3f0289957a-kube-api-access-l8pnk\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.407022 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.407004 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac436778-eb3c-42c3-8187-fe3f0289957a-cert\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.449687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.449633 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5x7p6\"" Nov 04 05:33:55.456860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.456835 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:33:55.469789 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.469761 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:55.505913 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.505880 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.506029 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.505943 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mhr\" (UniqueName: \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.521461 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.521407 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8pnk\" (UniqueName: \"kubernetes.io/projected/ac436778-eb3c-42c3-8187-fe3f0289957a-kube-api-access-l8pnk\") pod \"kserve-controller-manager-776cc66d47-tn2gh\" (UID: \"ac436778-eb3c-42c3-8187-fe3f0289957a\") " pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.551199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.551175 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Nov 04 05:33:55.558523 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.558494 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.573559 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.573539 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4mhr\" (UniqueName: \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\") pod \"llmisvc-controller-manager-869d5dcc5d-kmbhd\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.579338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.579317 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:33:55.700806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.700771 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:33:55.739564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.739528 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:33:55.967187 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:55.967163 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-tn2gh"] Nov 04 05:33:55.969809 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:33:55.969784 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac436778_eb3c_42c3_8187_fe3f0289957a.slice/crio-f9e902bdfcd86e43324b5d81869d5acb05ef437f1a44fe8c8c9369a52966cc8d WatchSource:0}: Error finding container f9e902bdfcd86e43324b5d81869d5acb05ef437f1a44fe8c8c9369a52966cc8d: Status 404 returned error can't find the container with id f9e902bdfcd86e43324b5d81869d5acb05ef437f1a44fe8c8c9369a52966cc8d Nov 04 05:33:56.085623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:56.085600 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:33:56.087579 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:33:56.087547 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7927c5e_2b84_49fb_a579_bdf0c2c3aade.slice/crio-00a9be571b63040a9a2699e9ea8b34634cac0b5da33dd5aac2b4274b9a696bc4 WatchSource:0}: Error finding container 00a9be571b63040a9a2699e9ea8b34634cac0b5da33dd5aac2b4274b9a696bc4: Status 404 returned error can't find the container with id 00a9be571b63040a9a2699e9ea8b34634cac0b5da33dd5aac2b4274b9a696bc4 Nov 04 05:33:56.428031 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:56.427946 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" event={"ID":"e7927c5e-2b84-49fb-a579-bdf0c2c3aade","Type":"ContainerStarted","Data":"00a9be571b63040a9a2699e9ea8b34634cac0b5da33dd5aac2b4274b9a696bc4"} Nov 04 05:33:56.429171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:56.429147 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" event={"ID":"ac436778-eb3c-42c3-8187-fe3f0289957a","Type":"ContainerStarted","Data":"f9e902bdfcd86e43324b5d81869d5acb05ef437f1a44fe8c8c9369a52966cc8d"} Nov 04 05:33:56.747264 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:33:56.747205 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:00.454305 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.454199 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" event={"ID":"ac436778-eb3c-42c3-8187-fe3f0289957a","Type":"ContainerStarted","Data":"09a70c235a8728381cd9464904fc01bd49886b8bf11147127857fcd0cefa0b7a"} Nov 04 05:34:00.454305 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.454287 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:34:00.455837 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.455815 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" event={"ID":"e7927c5e-2b84-49fb-a579-bdf0c2c3aade","Type":"ContainerStarted","Data":"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e"} Nov 04 05:34:00.455949 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.455938 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:34:00.489286 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:34:00.489253 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:34:00.583712 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.583660 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" podStartSLOduration=1.517104839 podStartE2EDuration="5.583649099s" totalImagesPullingTime="4.06654426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:00.583418969 +0000 UTC m=+378.147802269" watchObservedRunningTime="2026-11-04 05:34:00.583649099 +0000 UTC m=+378.148032398" Nov 04 05:34:00.713389 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:00.713289 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" podStartSLOduration=1.757548294 podStartE2EDuration="5.713275293s" totalImagesPullingTime="3.955726999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:00.712861721 +0000 UTC m=+378.277245021" watchObservedRunningTime="2026-11-04 05:34:00.713275293 +0000 UTC m=+378.277658590" Nov 04 05:34:01.888378 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:01.888345 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:02.085814 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:02.085781 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:02.303234 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:02.303172 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:02.512811 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:02.512772 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:02.712892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:02.712786 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:02.932538 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:02.932496 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:03.154430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:03.154384 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:05.868612 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:05.868568 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:06.077596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:06.077553 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:06.649171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:06.649133 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:06.895140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:06.895107 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:07.979132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:07.979100 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:08.204279 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:08.204212 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:08.410497 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:08.410409 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:20.652457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:20.652410 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:20.860144 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:20.860101 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:20.914437 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:20.914402 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:20.917767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:20.917745 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:20.981687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:20.981647 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:21.032783 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.032749 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6d6\" (UniqueName: \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\") pod \"authorino-6f6944bcdd-gb4fm\" (UID: \"f15a6274-78db-4e6a-88c3-c443235494fd\") " pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:21.134057 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.133968 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6d6\" (UniqueName: \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\") pod \"authorino-6f6944bcdd-gb4fm\" (UID: \"f15a6274-78db-4e6a-88c3-c443235494fd\") " pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:21.202376 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.202339 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6d6\" (UniqueName: \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\") pod \"authorino-6f6944bcdd-gb4fm\" (UID: \"f15a6274-78db-4e6a-88c3-c443235494fd\") " pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:21.230620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.230589 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:21.306206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.304467 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:21.331515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.331487 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:21.335806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.335788 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:21.397440 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.397362 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:21.436706 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.436656 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlpcf\" (UniqueName: \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\") pod \"authorino-bb989484b-kbg9f\" (UID: \"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9\") " pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:21.537992 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.537949 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlpcf\" (UniqueName: \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\") pod \"authorino-bb989484b-kbg9f\" (UID: \"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9\") " pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:21.605993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.605961 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlpcf\" (UniqueName: \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\") pod \"authorino-bb989484b-kbg9f\" (UID: \"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9\") " pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:21.616798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.616768 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:21.619503 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:34:21.619472 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf15a6274_78db_4e6a_88c3_c443235494fd.slice/crio-be0ebb50898ebbfd6d7794ade5296c858578b286f1cef5cecd7c4e6fb18041e7 WatchSource:0}: Error finding container be0ebb50898ebbfd6d7794ade5296c858578b286f1cef5cecd7c4e6fb18041e7: Status 404 returned error can't find the container with id be0ebb50898ebbfd6d7794ade5296c858578b286f1cef5cecd7c4e6fb18041e7 Nov 04 05:34:21.647766 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.647694 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:21.787742 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.787709 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:21.818159 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.818130 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:34:21.822668 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.822644 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:21.882772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.882741 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Nov 04 05:34:21.885685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.885657 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:34:21.943368 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.943280 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:21.943368 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:21.943332 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5v22\" (UniqueName: \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.034370 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.034338 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:22.035744 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:34:22.035717 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f8990b9_ef68_4c43_9861_2de3bbf1cbe9.slice/crio-27a938003d65062385fc00c3e0d60d4f9129765da3deb38243e116c97810b494 WatchSource:0}: Error finding container 27a938003d65062385fc00c3e0d60d4f9129765da3deb38243e116c97810b494: Status 404 returned error can't find the container with id 27a938003d65062385fc00c3e0d60d4f9129765da3deb38243e116c97810b494 Nov 04 05:34:22.044478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.044450 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.044568 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.044492 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5v22\" (UniqueName: \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.046712 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.046691 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.116897 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.116861 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5v22\" (UniqueName: \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\") pod \"authorino-ff4cf6dcb-nr9cw\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.134680 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.134654 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:34:22.524038 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.524010 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:34:22.527361 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:34:22.527336 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fc06917_869e_49dd_92f5_76652e6e7309.slice/crio-1cc41ee9e6f15ee1061f0c7506065b5aa348dde57535477a914a307e8c6c792e WatchSource:0}: Error finding container 1cc41ee9e6f15ee1061f0c7506065b5aa348dde57535477a914a307e8c6c792e: Status 404 returned error can't find the container with id 1cc41ee9e6f15ee1061f0c7506065b5aa348dde57535477a914a307e8c6c792e Nov 04 05:34:22.585236 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.585191 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" event={"ID":"f15a6274-78db-4e6a-88c3-c443235494fd","Type":"ContainerStarted","Data":"be0ebb50898ebbfd6d7794ade5296c858578b286f1cef5cecd7c4e6fb18041e7"} Nov 04 05:34:22.586432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.586407 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" event={"ID":"0fc06917-869e-49dd-92f5-76652e6e7309","Type":"ContainerStarted","Data":"1cc41ee9e6f15ee1061f0c7506065b5aa348dde57535477a914a307e8c6c792e"} Nov 04 05:34:22.587599 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:22.587574 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb989484b-kbg9f" event={"ID":"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9","Type":"ContainerStarted","Data":"27a938003d65062385fc00c3e0d60d4f9129765da3deb38243e116c97810b494"} Nov 04 05:34:23.594152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.594126 5104 generic.go:356] "Generic (PLEG): container finished" podID="fb571d04-7026-4946-83dc-b487e5231e5a" containerID="8a0bfe50827187ebcdaa9699c13abf16b0bfb6a83b587ec1abf1a3b41e31c45e" exitCode=0 Nov 04 05:34:23.594554 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.594194 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" event={"ID":"fb571d04-7026-4946-83dc-b487e5231e5a","Type":"ContainerDied","Data":"8a0bfe50827187ebcdaa9699c13abf16b0bfb6a83b587ec1abf1a3b41e31c45e"} Nov 04 05:34:23.594626 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.594595 5104 scope.go:122] "RemoveContainer" containerID="8a0bfe50827187ebcdaa9699c13abf16b0bfb6a83b587ec1abf1a3b41e31c45e" Nov 04 05:34:23.595838 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.595783 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:34:23.595951 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.595884 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:34:23.595951 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.595934 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:34:23.596081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.596034 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" podUID="f15a6274-78db-4e6a-88c3-c443235494fd" containerName="authorino" containerID="cri-o://61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7" gracePeriod=30 Nov 04 05:34:23.596140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.596037 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" event={"ID":"f15a6274-78db-4e6a-88c3-c443235494fd","Type":"ContainerStarted","Data":"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7"} Nov 04 05:34:23.847754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.847712 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:23.879967 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.879915 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" podStartSLOduration=2.253147836 podStartE2EDuration="3.879900552s" totalImagesPullingTime="1.626752716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:23.878516211 +0000 UTC m=+401.442899510" watchObservedRunningTime="2026-11-04 05:34:23.879900552 +0000 UTC m=+401.444283851" Nov 04 05:34:23.959553 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.959530 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\" (UniqueName: \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\") pod \"f15a6274-78db-4e6a-88c3-c443235494fd\" (UID: \"f15a6274-78db-4e6a-88c3-c443235494fd\") " Nov 04 05:34:23.961785 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:23.961750 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6" pod "f15a6274-78db-4e6a-88c3-c443235494fd" (UID: "f15a6274-78db-4e6a-88c3-c443235494fd"). InnerVolumeSpecName "kube-api-access-fm6d6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:34:24.060202 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.060171 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm6d6\" (UniqueName: \"kubernetes.io/projected/f15a6274-78db-4e6a-88c3-c443235494fd-kube-api-access-fm6d6\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:24.608539 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.608499 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" event={"ID":"0fc06917-869e-49dd-92f5-76652e6e7309","Type":"ContainerStarted","Data":"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae"} Nov 04 05:34:24.616243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.612928 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb989484b-kbg9f" event={"ID":"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9","Type":"ContainerStarted","Data":"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51"} Nov 04 05:34:24.616243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.613092 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-bb989484b-kbg9f" podUID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" containerName="authorino" containerID="cri-o://fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51" gracePeriod=30 Nov 04 05:34:24.616243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.615912 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" event={"ID":"fb571d04-7026-4946-83dc-b487e5231e5a","Type":"ContainerStarted","Data":"913160fff21dd683d216530b6c21e971357bead6c961123775b6534f7f7d9b6b"} Nov 04 05:34:24.622267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.617396 5104 generic.go:356] "Generic (PLEG): container finished" podID="f15a6274-78db-4e6a-88c3-c443235494fd" containerID="61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7" exitCode=0 Nov 04 05:34:24.622267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.617434 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" event={"ID":"f15a6274-78db-4e6a-88c3-c443235494fd","Type":"ContainerDied","Data":"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7"} Nov 04 05:34:24.622267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.617463 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" event={"ID":"f15a6274-78db-4e6a-88c3-c443235494fd","Type":"ContainerDied","Data":"be0ebb50898ebbfd6d7794ade5296c858578b286f1cef5cecd7c4e6fb18041e7"} Nov 04 05:34:24.622267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.617485 5104 scope.go:122] "RemoveContainer" containerID="61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7" Nov 04 05:34:24.622267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.617638 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-gb4fm" Nov 04 05:34:24.666894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.666861 5104 scope.go:122] "RemoveContainer" containerID="61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7" Nov 04 05:34:24.667317 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:34:24.667289 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7\": container with ID starting with 61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7 not found: ID does not exist" containerID="61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7" Nov 04 05:34:24.667452 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.667327 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7"} err="failed to get container status \"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7\": rpc error: code = NotFound desc = could not find container \"61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7\": container with ID starting with 61b831a442a162f034bf30e17d7d5a7ec465f1f55ef31328c222af12aca4a7b7 not found: ID does not exist" Nov 04 05:34:24.706564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.706529 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:34:24.707626 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.707601 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:24.707707 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.707654 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:24.769389 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.769330 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" podStartSLOduration=2.366302804 podStartE2EDuration="3.769312987s" totalImagesPullingTime="1.403010183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:24.758879898 +0000 UTC m=+402.323263208" watchObservedRunningTime="2026-11-04 05:34:24.769312987 +0000 UTC m=+402.333696286" Nov 04 05:34:24.769797 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.769778 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:34:24.769996 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.769974 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" podUID="8ce86675-48f4-4da4-b55f-604994e1ee3a" containerName="authorino" containerID="cri-o://0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a" gracePeriod=30 Nov 04 05:34:24.888145 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.887923 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:24.891950 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.891838 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-gb4fm"] Nov 04 05:34:24.975013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:24.974977 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:25.080034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.079470 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\" (UniqueName: \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\") pod \"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9\" (UID: \"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9\") " Nov 04 05:34:25.082689 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.082646 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf" pod "5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" (UID: "5f8990b9-ef68-4c43-9861-2de3bbf1cbe9"). InnerVolumeSpecName "kube-api-access-rlpcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:34:25.085091 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.085070 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:34:25.180271 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.180152 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\" (UniqueName: \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\") pod \"8ce86675-48f4-4da4-b55f-604994e1ee3a\" (UID: \"8ce86675-48f4-4da4-b55f-604994e1ee3a\") " Nov 04 05:34:25.180497 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.180478 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlpcf\" (UniqueName: \"kubernetes.io/projected/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9-kube-api-access-rlpcf\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:25.182376 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.182345 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts" pod "8ce86675-48f4-4da4-b55f-604994e1ee3a" (UID: "8ce86675-48f4-4da4-b55f-604994e1ee3a"). InnerVolumeSpecName "kube-api-access-8stts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:34:25.193527 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.193478 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-bb989484b-kbg9f" podStartSLOduration=2.67151126 podStartE2EDuration="4.193466109s" totalImagesPullingTime="1.521954849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:34:25.192071089 +0000 UTC m=+402.756454389" watchObservedRunningTime="2026-11-04 05:34:25.193466109 +0000 UTC m=+402.757849426" Nov 04 05:34:25.281659 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.281629 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8stts\" (UniqueName: \"kubernetes.io/projected/8ce86675-48f4-4da4-b55f-604994e1ee3a-kube-api-access-8stts\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:34:25.623761 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.623715 5104 generic.go:356] "Generic (PLEG): container finished" podID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" containerID="fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51" exitCode=0 Nov 04 05:34:25.624263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.623773 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb989484b-kbg9f" Nov 04 05:34:25.624263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.623780 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb989484b-kbg9f" event={"ID":"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9","Type":"ContainerDied","Data":"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51"} Nov 04 05:34:25.624263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.623818 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb989484b-kbg9f" event={"ID":"5f8990b9-ef68-4c43-9861-2de3bbf1cbe9","Type":"ContainerDied","Data":"27a938003d65062385fc00c3e0d60d4f9129765da3deb38243e116c97810b494"} Nov 04 05:34:25.624263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.623840 5104 scope.go:122] "RemoveContainer" containerID="fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51" Nov 04 05:34:25.626614 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.626586 5104 generic.go:356] "Generic (PLEG): container finished" podID="8ce86675-48f4-4da4-b55f-604994e1ee3a" containerID="0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a" exitCode=0 Nov 04 05:34:25.626738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.626618 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" Nov 04 05:34:25.626738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.626674 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" event={"ID":"8ce86675-48f4-4da4-b55f-604994e1ee3a","Type":"ContainerDied","Data":"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a"} Nov 04 05:34:25.626738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.626704 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-t5wpr" event={"ID":"8ce86675-48f4-4da4-b55f-604994e1ee3a","Type":"ContainerDied","Data":"6b8e4582e38a01a1a594f7291381657c7bc08fc7b385d25962fee12e9f59086f"} Nov 04 05:34:25.656064 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.655803 5104 scope.go:122] "RemoveContainer" containerID="fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51" Nov 04 05:34:25.659138 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:34:25.659107 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51\": container with ID starting with fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51 not found: ID does not exist" containerID="fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51" Nov 04 05:34:25.659287 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.659151 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51"} err="failed to get container status \"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51\": rpc error: code = NotFound desc = could not find container \"fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51\": container with ID starting with fbb71e25b850323d4a18c9f3c2c3b0cfa86ceab60260b0690c084abd2dab7b51 not found: ID does not exist" Nov 04 05:34:25.659287 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.659180 5104 scope.go:122] "RemoveContainer" containerID="0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a" Nov 04 05:34:25.670132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.670115 5104 scope.go:122] "RemoveContainer" containerID="0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a" Nov 04 05:34:25.670536 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:34:25.670514 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a\": container with ID starting with 0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a not found: ID does not exist" containerID="0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a" Nov 04 05:34:25.670617 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.670543 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a"} err="failed to get container status \"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a\": rpc error: code = NotFound desc = could not find container \"0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a\": container with ID starting with 0577ce52579aa3aeb83e07c847488d704f8d651f2afd8baec8c8fb6ccff7782a not found: ID does not exist" Nov 04 05:34:25.705621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.705594 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:25.705745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.705639 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:25.818947 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.818917 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:34:25.823207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.823181 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-t5wpr"] Nov 04 05:34:25.959414 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.959316 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:25.960463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:25.960437 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bb989484b-kbg9f"] Nov 04 05:34:26.563540 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:26.563509 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" path="/var/lib/kubelet/pods/5f8990b9-ef68-4c43-9861-2de3bbf1cbe9/volumes" Nov 04 05:34:26.563882 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:26.563868 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ce86675-48f4-4da4-b55f-604994e1ee3a" path="/var/lib/kubelet/pods/8ce86675-48f4-4da4-b55f-604994e1ee3a/volumes" Nov 04 05:34:26.564182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:26.564170 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f15a6274-78db-4e6a-88c3-c443235494fd" path="/var/lib/kubelet/pods/f15a6274-78db-4e6a-88c3-c443235494fd/volumes" Nov 04 05:34:26.705541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:26.705504 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:26.705926 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:26.705560 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:27.705169 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:27.705132 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:27.705388 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:27.705193 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:28.414323 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:28.414267 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:28.618988 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:28.618952 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:34:28.706168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:28.706081 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:28.706168 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:28.706134 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:29.454246 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:29.453790 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:34:29.705766 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:29.705689 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-tmnh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 04 05:34:29.705904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:29.705764 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" podUID="fb571d04-7026-4946-83dc-b487e5231e5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 04 05:34:30.491136 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:34:30.491096 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:34:30.708758 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:30.708727 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:34:30.709887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:30.709867 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-tmnh8" Nov 04 05:34:31.463098 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:31.463067 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:34:31.465945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:34:31.465921 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-tn2gh" Nov 04 05:35:00.489929 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:35:00.489890 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:35:02.645187 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:02.645148 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:35:06.142071 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142028 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-flnsj"] Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142419 5104 cpu_manager.go:414] "containerMap: removing container" podUID="8ce86675-48f4-4da4-b55f-604994e1ee3a" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142431 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ce86675-48f4-4da4-b55f-604994e1ee3a" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142447 5104 cpu_manager.go:414] "containerMap: removing container" podUID="f15a6274-78db-4e6a-88c3-c443235494fd" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142452 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f15a6274-78db-4e6a-88c3-c443235494fd" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142477 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142483 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" containerName="authorino" Nov 04 05:35:06.142535 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142538 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f8990b9-ef68-4c43-9861-2de3bbf1cbe9" containerName="authorino" Nov 04 05:35:06.142754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142552 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ce86675-48f4-4da4-b55f-604994e1ee3a" containerName="authorino" Nov 04 05:35:06.142754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.142560 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="f15a6274-78db-4e6a-88c3-c443235494fd" containerName="authorino" Nov 04 05:35:06.145695 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.145678 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.209574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.209543 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-flnsj"] Nov 04 05:35:06.225685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.225659 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g22r\" (UniqueName: \"kubernetes.io/projected/6830b443-72e9-487f-8754-69dc783b70fd-kube-api-access-7g22r\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.225800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.225693 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6830b443-72e9-487f-8754-69dc783b70fd-cert\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.326517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.326478 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g22r\" (UniqueName: \"kubernetes.io/projected/6830b443-72e9-487f-8754-69dc783b70fd-kube-api-access-7g22r\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.326517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.326517 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6830b443-72e9-487f-8754-69dc783b70fd-cert\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.328865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.328838 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6830b443-72e9-487f-8754-69dc783b70fd-cert\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.394690 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.394633 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g22r\" (UniqueName: \"kubernetes.io/projected/6830b443-72e9-487f-8754-69dc783b70fd-kube-api-access-7g22r\") pod \"llmisvc-controller-manager-6466c554d8-flnsj\" (UID: \"6830b443-72e9-487f-8754-69dc783b70fd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.457763 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.457738 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:06.845034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:06.845007 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-flnsj"] Nov 04 05:35:06.847509 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:35:06.847483 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6830b443_72e9_487f_8754_69dc783b70fd.slice/crio-55bc4c0b9b9e9a711453e278a7a3efbcda3e64a68d19dc0873e38c8263551555 WatchSource:0}: Error finding container 55bc4c0b9b9e9a711453e278a7a3efbcda3e64a68d19dc0873e38c8263551555: Status 404 returned error can't find the container with id 55bc4c0b9b9e9a711453e278a7a3efbcda3e64a68d19dc0873e38c8263551555 Nov 04 05:35:07.855118 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:07.855078 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" event={"ID":"6830b443-72e9-487f-8754-69dc783b70fd","Type":"ContainerStarted","Data":"65e44aef7a6594beb5267ce843997ada04fa3d0d6fcc77353bad9b04536380f0"} Nov 04 05:35:07.855490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:07.855134 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" event={"ID":"6830b443-72e9-487f-8754-69dc783b70fd","Type":"ContainerStarted","Data":"55bc4c0b9b9e9a711453e278a7a3efbcda3e64a68d19dc0873e38c8263551555"} Nov 04 05:35:07.855490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:07.855181 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:07.990406 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:07.990343 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" podStartSLOduration=1.135673538 podStartE2EDuration="1.990329981s" totalImagesPullingTime="854.656443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:35:07.988934849 +0000 UTC m=+445.553318147" watchObservedRunningTime="2026-11-04 05:35:07.990329981 +0000 UTC m=+445.554713280" Nov 04 05:35:13.116002 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:13.115968 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:35:30.490725 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:35:30.490626 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:35:38.862589 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:38.862554 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-flnsj" Nov 04 05:35:39.051201 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.051166 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:35:39.051468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.051418 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" podUID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" containerName="manager" containerID="cri-o://8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e" gracePeriod=30 Nov 04 05:35:39.298830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.298804 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:35:39.385118 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.385086 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\" (UniqueName: \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\") pod \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " Nov 04 05:35:39.385293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.385246 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\" (UniqueName: \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\") pod \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\" (UID: \"e7927c5e-2b84-49fb-a579-bdf0c2c3aade\") " Nov 04 05:35:39.387124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.387099 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert" pod "e7927c5e-2b84-49fb-a579-bdf0c2c3aade" (UID: "e7927c5e-2b84-49fb-a579-bdf0c2c3aade"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:35:39.387258 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.387239 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr" pod "e7927c5e-2b84-49fb-a579-bdf0c2c3aade" (UID: "e7927c5e-2b84-49fb-a579-bdf0c2c3aade"). InnerVolumeSpecName "kube-api-access-m4mhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:35:39.486334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.486305 5104 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:35:39.486334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.486331 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4mhr\" (UniqueName: \"kubernetes.io/projected/e7927c5e-2b84-49fb-a579-bdf0c2c3aade-kube-api-access-m4mhr\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:35:39.963077 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:39.962993 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:35:40.010878 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.010844 5104 generic.go:356] "Generic (PLEG): container finished" podID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" containerID="8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e" exitCode=0 Nov 04 05:35:40.011030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.010891 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" Nov 04 05:35:40.011030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.010923 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" event={"ID":"e7927c5e-2b84-49fb-a579-bdf0c2c3aade","Type":"ContainerDied","Data":"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e"} Nov 04 05:35:40.011030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.010977 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd" event={"ID":"e7927c5e-2b84-49fb-a579-bdf0c2c3aade","Type":"ContainerDied","Data":"00a9be571b63040a9a2699e9ea8b34634cac0b5da33dd5aac2b4274b9a696bc4"} Nov 04 05:35:40.011030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.011001 5104 scope.go:122] "RemoveContainer" containerID="8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e" Nov 04 05:35:40.021945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.021924 5104 scope.go:122] "RemoveContainer" containerID="8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e" Nov 04 05:35:40.022276 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:35:40.022256 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e\": container with ID starting with 8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e not found: ID does not exist" containerID="8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e" Nov 04 05:35:40.022352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.022285 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e"} err="failed to get container status \"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e\": rpc error: code = NotFound desc = could not find container \"8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e\": container with ID starting with 8e1057783c3e84123066c89de84047f73c19b40148152b8d03f4cdfb8c5fe00e not found: ID does not exist" Nov 04 05:35:40.176380 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.176346 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:35:40.205546 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.205513 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:35:40.260924 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.260897 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-kmbhd"] Nov 04 05:35:40.549206 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:35:40.549131 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" path="/var/lib/kubelet/pods/e7927c5e-2b84-49fb-a579-bdf0c2c3aade/volumes" Nov 04 05:36:00.490434 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:36:00.490395 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:36:02.451087 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:02.451053 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:03.021170 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:03.021135 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:06.086445 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:06.086409 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:08.065665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:08.065581 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:08.270571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:08.270519 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:09.318641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:09.318605 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:16.555125 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:16.555090 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:21.603291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:21.603236 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:23.250475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:23.250447 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:27.455470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:27.455428 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:30.493796 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:36:30.493757 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:36:31.059021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:31.058983 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:34.479593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:34.479557 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:36:47.964884 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:47.964848 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl"] Nov 04 05:36:47.965365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:47.965265 5104 cpu_manager.go:414] "containerMap: removing container" podUID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" containerName="manager" Nov 04 05:36:47.965365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:47.965282 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" containerName="manager" Nov 04 05:36:47.965365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:47.965358 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7927c5e-2b84-49fb-a579-bdf0c2c3aade" containerName="manager" Nov 04 05:36:47.968626 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:47.968607 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.048901 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.048868 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0935188-2146-4564-9576-30cd88fae725-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049050 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.048914 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0935188-2146-4564-9576-30cd88fae725-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049050 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049010 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049067 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfgk5\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-kube-api-access-jfgk5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049109 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049133 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049346 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049233 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049346 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049262 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.049346 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.049285 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150034 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150074 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150104 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150142 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0935188-2146-4564-9576-30cd88fae725-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150174 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0935188-2146-4564-9576-30cd88fae725-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150233 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150272 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfgk5\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-kube-api-access-jfgk5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150310 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150335 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150514 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150758 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150701 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.150964 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150942 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.151010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.150988 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0935188-2146-4564-9576-30cd88fae725-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.151043 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.151012 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.152529 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.152501 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0935188-2146-4564-9576-30cd88fae725-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.152655 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.152629 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0935188-2146-4564-9576-30cd88fae725-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.192699 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.192677 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl"] Nov 04 05:36:48.217602 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.217575 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.217880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.217857 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfgk5\" (UniqueName: \"kubernetes.io/projected/b0935188-2146-4564-9576-30cd88fae725-kube-api-access-jfgk5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hrddl\" (UID: \"b0935188-2146-4564-9576-30cd88fae725\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.281847 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.281815 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:48.726128 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.726103 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl"] Nov 04 05:36:48.728451 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:36:48.728424 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0935188_2146_4564_9576_30cd88fae725.slice/crio-91b94e5b824b5e88da99f7515d22cb9b83bbc780ec31dc01373f348fde7e72f7 WatchSource:0}: Error finding container 91b94e5b824b5e88da99f7515d22cb9b83bbc780ec31dc01373f348fde7e72f7: Status 404 returned error can't find the container with id 91b94e5b824b5e88da99f7515d22cb9b83bbc780ec31dc01373f348fde7e72f7 Nov 04 05:36:48.730465 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.730432 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:36:48.730560 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.730497 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:36:48.730560 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:48.730526 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:36:49.372917 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:49.372875 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" event={"ID":"b0935188-2146-4564-9576-30cd88fae725","Type":"ContainerStarted","Data":"49247118941f5b85f13cb940dad80999f596518d8385663baefa484896e3ca67"} Nov 04 05:36:49.373515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:49.372943 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" event={"ID":"b0935188-2146-4564-9576-30cd88fae725","Type":"ContainerStarted","Data":"91b94e5b824b5e88da99f7515d22cb9b83bbc780ec31dc01373f348fde7e72f7"} Nov 04 05:36:49.564596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:49.564537 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podStartSLOduration=2.564522765 podStartE2EDuration="2.564522765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:49.562785378 +0000 UTC m=+547.127168678" watchObservedRunningTime="2026-11-04 05:36:49.564522765 +0000 UTC m=+547.128906064" Nov 04 05:36:50.282964 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:50.282929 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:50.284391 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:50.284361 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:50.284517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:50.284409 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:51.282458 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:51.282374 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:51.282458 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:51.282434 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:52.282462 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:52.282426 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:52.282857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:52.282481 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:53.282203 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:53.282164 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:53.282422 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:53.282246 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:54.282405 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:54.282364 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:54.282804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:54.282433 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:55.282875 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:55.282845 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:55.283261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:55.282908 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:56.282611 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:56.282573 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hrddl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 05:36:56.282785 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:56.282625 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" podUID="b0935188-2146-4564-9576-30cd88fae725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 05:36:57.285807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:57.285778 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:57.286266 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:57.286251 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:36:57.286873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:36:57.286851 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hrddl" Nov 04 05:37:00.489970 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:37:00.489933 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:37:03.417541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:03.417507 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:04.533264 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.533211 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-667656475-7s2pb"] Nov 04 05:37:04.537156 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.537134 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.593384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.593360 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qdq\" (UniqueName: \"kubernetes.io/projected/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-kube-api-access-s9qdq\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.593528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.593409 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-tls-cert\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.600159 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.600136 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-667656475-7s2pb"] Nov 04 05:37:04.695025 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.694980 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qdq\" (UniqueName: \"kubernetes.io/projected/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-kube-api-access-s9qdq\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.695165 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.695103 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-tls-cert\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.697604 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.697577 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-tls-cert\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.761783 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.761744 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qdq\" (UniqueName: \"kubernetes.io/projected/f9fe744b-4bbe-4b56-a3a7-a10afc8688a9-kube-api-access-s9qdq\") pod \"authorino-667656475-7s2pb\" (UID: \"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9\") " pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:04.849751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:04.849670 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-667656475-7s2pb" Nov 04 05:37:05.244174 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:05.244147 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-667656475-7s2pb"] Nov 04 05:37:05.246468 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:37:05.246439 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9fe744b_4bbe_4b56_a3a7_a10afc8688a9.slice/crio-db365a66d53b38b9c2dc66b27b589feeb0f45f70fa0507d7b209ad5ef2f8840e WatchSource:0}: Error finding container db365a66d53b38b9c2dc66b27b589feeb0f45f70fa0507d7b209ad5ef2f8840e: Status 404 returned error can't find the container with id db365a66d53b38b9c2dc66b27b589feeb0f45f70fa0507d7b209ad5ef2f8840e Nov 04 05:37:05.461963 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:05.461927 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-667656475-7s2pb" event={"ID":"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9","Type":"ContainerStarted","Data":"db365a66d53b38b9c2dc66b27b589feeb0f45f70fa0507d7b209ad5ef2f8840e"} Nov 04 05:37:05.471602 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:05.471578 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:07.473198 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:07.473169 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-667656475-7s2pb" event={"ID":"f9fe744b-4bbe-4b56-a3a7-a10afc8688a9","Type":"ContainerStarted","Data":"6aaf0f8638ee143e778ff2b87249e2707ef740b0b5154ae3db458c0a13c97196"} Nov 04 05:37:07.600798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:07.600749 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-667656475-7s2pb" podStartSLOduration=1.672130089 podStartE2EDuration="3.600735883s" totalImagesPullingTime="1.928605794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:07.599434081 +0000 UTC m=+565.163817381" watchObservedRunningTime="2026-11-04 05:37:07.600735883 +0000 UTC m=+565.165119182" Nov 04 05:37:07.656786 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:07.656748 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:37:07.657338 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:07.656980 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" podUID="0fc06917-869e-49dd-92f5-76652e6e7309" containerName="authorino" containerID="cri-o://c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae" gracePeriod=30 Nov 04 05:37:07.921546 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:07.921513 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:37:08.027062 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.026968 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\" (UniqueName: \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\") pod \"0fc06917-869e-49dd-92f5-76652e6e7309\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " Nov 04 05:37:08.027253 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.027096 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\" (UniqueName: \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\") pod \"0fc06917-869e-49dd-92f5-76652e6e7309\" (UID: \"0fc06917-869e-49dd-92f5-76652e6e7309\") " Nov 04 05:37:08.029848 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.029819 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22" pod "0fc06917-869e-49dd-92f5-76652e6e7309" (UID: "0fc06917-869e-49dd-92f5-76652e6e7309"). InnerVolumeSpecName "kube-api-access-j5v22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:37:08.041262 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.041232 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert" pod "0fc06917-869e-49dd-92f5-76652e6e7309" (UID: "0fc06917-869e-49dd-92f5-76652e6e7309"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:37:08.128800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.128763 5104 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fc06917-869e-49dd-92f5-76652e6e7309-tls-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:08.128800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.128797 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5v22\" (UniqueName: \"kubernetes.io/projected/0fc06917-869e-49dd-92f5-76652e6e7309-kube-api-access-j5v22\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:37:08.480291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.480257 5104 generic.go:356] "Generic (PLEG): container finished" podID="0fc06917-869e-49dd-92f5-76652e6e7309" containerID="c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae" exitCode=0 Nov 04 05:37:08.480767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.480324 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" Nov 04 05:37:08.480767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.480350 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" event={"ID":"0fc06917-869e-49dd-92f5-76652e6e7309","Type":"ContainerDied","Data":"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae"} Nov 04 05:37:08.480767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.480407 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff4cf6dcb-nr9cw" event={"ID":"0fc06917-869e-49dd-92f5-76652e6e7309","Type":"ContainerDied","Data":"1cc41ee9e6f15ee1061f0c7506065b5aa348dde57535477a914a307e8c6c792e"} Nov 04 05:37:08.480767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.480420 5104 scope.go:122] "RemoveContainer" containerID="c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae" Nov 04 05:37:08.489317 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.489298 5104 scope.go:122] "RemoveContainer" containerID="c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae" Nov 04 05:37:08.489618 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:37:08.489599 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae\": container with ID starting with c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae not found: ID does not exist" containerID="c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae" Nov 04 05:37:08.489687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.489627 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae"} err="failed to get container status \"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae\": rpc error: code = NotFound desc = could not find container \"c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae\": container with ID starting with c6bdaa26e3b859c99ebe2a16222acef0c415b0dbf485b275137293d9780105ae not found: ID does not exist" Nov 04 05:37:08.661980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.661935 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:37:08.666316 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:08.666284 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-ff4cf6dcb-nr9cw"] Nov 04 05:37:10.549597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:10.549557 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fc06917-869e-49dd-92f5-76652e6e7309" path="/var/lib/kubelet/pods/0fc06917-869e-49dd-92f5-76652e6e7309/volumes" Nov 04 05:37:14.934276 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:14.934246 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:15.154294 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:15.154248 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:30.450284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:30.450248 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:30.490680 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:37:30.490650 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:37:30.673269 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:30.673167 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:33.349499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.349466 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:37:33.349887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.349846 5104 cpu_manager.go:414] "containerMap: removing container" podUID="0fc06917-869e-49dd-92f5-76652e6e7309" containerName="authorino" Nov 04 05:37:33.349887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.349858 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fc06917-869e-49dd-92f5-76652e6e7309" containerName="authorino" Nov 04 05:37:33.349959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.349916 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fc06917-869e-49dd-92f5-76652e6e7309" containerName="authorino" Nov 04 05:37:33.353050 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.353030 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443655 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443629 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443674 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443696 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr2hm\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443772 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443912 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443820 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443912 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443856 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443976 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443927 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.443976 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443957 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.444040 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.443978 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.544995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.544961 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.544995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.544998 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545017 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545136 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545197 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545246 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr2hm\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545275 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545336 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545382 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545730 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545634 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545730 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545697 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545849 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545773 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.545904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545855 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.546009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.545985 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.547309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.547282 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.547509 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.547491 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.581236 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.581189 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:37:33.612067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.612000 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.616041 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.616011 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr2hm\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5gwkz\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:33.665122 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:33.665094 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:34.108551 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.108521 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:37:34.111014 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:37:34.110989 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f091441_e5dc_4e75_9939_9ff758997808.slice/crio-0a34c21e15fbbf297629a55ea167c5859971aaaa3726e0e6226f48f32993dc40 WatchSource:0}: Error finding container 0a34c21e15fbbf297629a55ea167c5859971aaaa3726e0e6226f48f32993dc40: Status 404 returned error can't find the container with id 0a34c21e15fbbf297629a55ea167c5859971aaaa3726e0e6226f48f32993dc40 Nov 04 05:37:34.112884 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.112849 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:37:34.112959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.112914 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:37:34.112959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.112948 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:37:34.614462 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.614425 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" event={"ID":"5f091441-e5dc-4e75-9939-9ff758997808","Type":"ContainerStarted","Data":"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c"} Nov 04 05:37:34.614857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.614487 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" event={"ID":"5f091441-e5dc-4e75-9939-9ff758997808","Type":"ContainerStarted","Data":"0a34c21e15fbbf297629a55ea167c5859971aaaa3726e0e6226f48f32993dc40"} Nov 04 05:37:34.665356 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.665310 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:34.803726 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:34.803673 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podStartSLOduration=1.8036590129999999 podStartE2EDuration="1.803659013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:34.801914054 +0000 UTC m=+592.366297354" watchObservedRunningTime="2026-11-04 05:37:34.803659013 +0000 UTC m=+592.368042311" Nov 04 05:37:35.666961 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:35.666924 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:35.667467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:35.666994 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:36.665892 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:36.665855 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:36.666104 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:36.665914 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:37.665705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:37.665672 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:37.666101 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:37.665721 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:38.666331 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:38.666293 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:38.666756 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:38.666351 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:39.666356 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:39.666321 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:39.666755 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:39.666376 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:40.665453 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:40.665417 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:40.665649 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:40.665477 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:41.666025 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:41.665983 5104 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5gwkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Nov 04 05:37:41.666408 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:41.666053 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Nov 04 05:37:41.874407 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:41.874378 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:42.400508 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:42.400476 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:42.669614 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:42.669557 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:42.669952 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:42.669933 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:42.670694 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:42.670678 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:37:44.118504 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:44.118463 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:44.341957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:44.341924 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:45.871425 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:45.871392 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:46.084894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:46.084862 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:59.001400 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:59.001366 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:37:59.214079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:37:59.214039 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:00.490383 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:38:00.490350 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:38:03.783887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:03.783849 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:30.490405 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:38:30.490325 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:38:35.606480 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:35.606425 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:35.828096 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:35.828045 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:36.256809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:36.256772 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:36.469132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:36.469102 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:36.681438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:36.681343 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:36.900277 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:36.900213 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.115107 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.115037 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.331098 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.331051 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.540353 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.540318 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.784341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.784304 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.785242 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.785204 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:38:37.789474 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.789450 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.847908 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.847868 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-901a00-openshift-default-dockercfg-txfc4\"" Nov 04 05:38:37.889867 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.889828 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.889868 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.889941 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.889967 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.889991 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.890167 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwnzk\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890282 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.890207 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890282 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.890253 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.890364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.890349 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.905663 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.905639 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:38:37.990759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990727 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwnzk\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990761 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990802 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990831 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990884 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990908 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990947 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990972 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.990995 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.990994 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.991385 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991330 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.991433 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991373 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:37.991476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991430 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.991766 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991738 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.991895 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991781 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.991986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.991964 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.993638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.993609 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:37.994085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:37.994065 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:38.057778 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.057696 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:38.057928 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.057818 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwnzk\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:38.103705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.103675 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:38.201422 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.201278 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:38.411530 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.411438 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:38.556625 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.556602 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:38:38.558538 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:38:38.558508 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8109cf5d_32e0_42da_9369_ba00a8207a8c.slice/crio-b6ebf7b33e1e01d7dc991a8d5605c2ede2519eb2d1bcdc7bee7c4f5ce3ddb117 WatchSource:0}: Error finding container b6ebf7b33e1e01d7dc991a8d5605c2ede2519eb2d1bcdc7bee7c4f5ce3ddb117: Status 404 returned error can't find the container with id b6ebf7b33e1e01d7dc991a8d5605c2ede2519eb2d1bcdc7bee7c4f5ce3ddb117 Nov 04 05:38:38.560701 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.560662 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:38:38.560805 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.560743 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:38:38.560805 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.560791 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:38:38.657584 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.657554 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:38.950085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.950048 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerStarted","Data":"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d"} Nov 04 05:38:38.950587 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:38.950115 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerStarted","Data":"b6ebf7b33e1e01d7dc991a8d5605c2ede2519eb2d1bcdc7bee7c4f5ce3ddb117"} Nov 04 05:38:39.085549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:39.085494 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podStartSLOduration=2.085480766 podStartE2EDuration="2.085480766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:39.082698753 +0000 UTC m=+656.647082052" watchObservedRunningTime="2026-11-04 05:38:39.085480766 +0000 UTC m=+656.649864121" Nov 04 05:38:39.104478 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:39.104453 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:39.109341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:39.109319 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:39.955540 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:39.955500 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:39.956480 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:39.956463 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:38:40.706466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:40.706427 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:40.927551 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:40.927520 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:41.456008 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:41.455973 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:41.674708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:41.674625 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:41.891707 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:41.891670 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:42.621714 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:42.621676 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:42.852104 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:42.852071 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:43.068959 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:43.068929 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:43.551439 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:43.551407 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:43.786955 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:43.786922 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:43.997475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:43.997444 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:45.993488 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:45.993455 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:46.206975 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:46.206944 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:46.571657 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:46.571625 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:46.792646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:46.792616 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:47.286062 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:47.286032 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:47.513522 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:47.513488 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:48.091287 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:48.091248 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:48.308060 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:48.308023 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:49.104617 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:49.104582 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:49.314728 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:49.314697 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:49.540297 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:49.540265 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:51.316983 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:51.316946 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:51.522421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:51.522388 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:53.899825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:53.899783 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:54.108579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:54.108546 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:54.315063 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:54.315035 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:59.411444 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:59.411392 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:59.622895 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:59.622861 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:38:59.836629 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:38:59.836598 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:00.043669 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:00.043633 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:00.490260 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:00.490208 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:39:04.155893 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:04.155855 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:04.757367 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:04.757330 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:04.967601 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:04.967570 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:05.030823 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.030746 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:39:05.034076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.034055 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.141178 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.141147 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.141360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.141187 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.141360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.141298 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc289\" (UniqueName: \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.141441 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.141417 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.147734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.147712 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-098178-serving-cert\"" Nov 04 05:39:05.147857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.147781 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 05:39:05.147899 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.147851 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8787b\"" Nov 04 05:39:05.149908 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.149887 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:39:05.242244 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.242204 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.242665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.242272 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc289\" (UniqueName: \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.242665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.242339 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.242665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.242405 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.242665 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:05.242519 5104 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-serving-cert: secret "maas-api-e2e-worker-w1-098178-serving-cert" not found Nov 04 05:39:05.242665 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:05.242607 5104 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls podName:b25581a3-ef28-4b8e-a197-1d64ffc5c32f nodeName:}" failed. No retries permitted until 2026-11-04 05:39:05.742586706 +0000 UTC m=+683.306969986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls") pod "maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" (UID: "b25581a3-ef28-4b8e-a197-1d64ffc5c32f") : secret "maas-api-e2e-worker-w1-098178-serving-cert" not found Nov 04 05:39:05.242968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.242947 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.244823 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.244797 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.307592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.307488 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:05.313746 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.313685 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc289\" (UniqueName: \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.524738 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.524706 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:05.738546 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.738512 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:05.747689 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.747662 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.750078 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.750052 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-098178-64854f4957-ttb4q\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:05.947817 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:05.947780 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:06.333274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:06.333250 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:39:06.336458 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:39:06.336428 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25581a3_ef28_4b8e_a197_1d64ffc5c32f.slice/crio-edce7c5e3ca6160bb90e5d6f2c894edb5bb6e973e42f5a43bd000eee7d16e089 WatchSource:0}: Error finding container edce7c5e3ca6160bb90e5d6f2c894edb5bb6e973e42f5a43bd000eee7d16e089: Status 404 returned error can't find the container with id edce7c5e3ca6160bb90e5d6f2c894edb5bb6e973e42f5a43bd000eee7d16e089 Nov 04 05:39:06.337993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:06.337970 5104 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:39:07.102752 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:07.102711 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" event={"ID":"b25581a3-ef28-4b8e-a197-1d64ffc5c32f","Type":"ContainerStarted","Data":"edce7c5e3ca6160bb90e5d6f2c894edb5bb6e973e42f5a43bd000eee7d16e089"} Nov 04 05:39:07.531576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:07.531541 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:07.747826 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:07.747730 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:10.120778 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.120738 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" event={"ID":"b25581a3-ef28-4b8e-a197-1d64ffc5c32f","Type":"ContainerStarted","Data":"f0c899ae555a776151616d7612d58ed7aeaa426aaa6303d509adf0ac351ef001"} Nov 04 05:39:10.121192 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.120804 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:10.258770 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.258705 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" podStartSLOduration=3.394906332 podStartE2EDuration="6.258686583s" totalImagesPullingTime="2.863780251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:10.255803909 +0000 UTC m=+687.820187209" watchObservedRunningTime="2026-11-04 05:39:10.258686583 +0000 UTC m=+687.823069883" Nov 04 05:39:10.377460 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.377379 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:10.592129 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.592098 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:10.965152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.965115 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:39:10.968794 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:10.968773 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.002747 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.002714 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.002898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.002754 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.002898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.002866 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.002980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.002915 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgw57\" (UniqueName: \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.064875 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.064843 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:39:11.074927 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.074892 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-283815-serving-cert\"" Nov 04 05:39:11.104363 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.104328 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgw57\" (UniqueName: \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.104520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.104385 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.104520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.104415 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.104520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.104473 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.104646 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:11.104546 5104 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-serving-cert: secret "maas-api-e2e-worker-w5-283815-serving-cert" not found Nov 04 05:39:11.104646 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:11.104604 5104 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls podName:f2bc7488-ea75-4fa0-aec6-e0e5f1367dea nodeName:}" failed. No retries permitted until 2026-11-04 05:39:11.604589603 +0000 UTC m=+689.168972880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls") pod "maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" (UID: "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea") : secret "maas-api-e2e-worker-w5-283815-serving-cert" not found Nov 04 05:39:11.105195 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.105171 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.106983 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.106956 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.171831 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.171800 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgw57\" (UniqueName: \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.268399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.268347 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:11.481258 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.481164 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:11.610159 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.610068 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.612402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.612384 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") pod \"maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:11.696320 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.696246 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:11.882733 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:11.882650 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:12.296957 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:12.296928 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:39:12.299027 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:39:12.299003 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bc7488_ea75_4fa0_aec6_e0e5f1367dea.slice/crio-ec4138b8099e79c6a7135e6ed0b337a6d59f68ea8ab56466a4f5f0244dfccc87 WatchSource:0}: Error finding container ec4138b8099e79c6a7135e6ed0b337a6d59f68ea8ab56466a4f5f0244dfccc87: Status 404 returned error can't find the container with id ec4138b8099e79c6a7135e6ed0b337a6d59f68ea8ab56466a4f5f0244dfccc87 Nov 04 05:39:13.139741 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:13.139700 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" event={"ID":"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea","Type":"ContainerStarted","Data":"ec4138b8099e79c6a7135e6ed0b337a6d59f68ea8ab56466a4f5f0244dfccc87"} Nov 04 05:39:14.147044 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:14.147003 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" event={"ID":"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea","Type":"ContainerStarted","Data":"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f"} Nov 04 05:39:14.147429 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:14.147123 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:14.278643 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:14.278591 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" podStartSLOduration=3.252384872 podStartE2EDuration="4.278576148s" totalImagesPullingTime="1.026191276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:14.276169474 +0000 UTC m=+691.840552773" watchObservedRunningTime="2026-11-04 05:39:14.278576148 +0000 UTC m=+691.842959447" Nov 04 05:39:16.134815 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.134787 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:39:16.159156 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.159125 5104 generic.go:356] "Generic (PLEG): container finished" podID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerID="67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d" exitCode=0 Nov 04 05:39:16.159299 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.159198 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerDied","Data":"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d"} Nov 04 05:39:16.159545 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.159532 5104 scope.go:122] "RemoveContainer" containerID="67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d" Nov 04 05:39:16.160729 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.160703 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:39:16.160781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.160772 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:39:16.160817 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.160807 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:39:16.907397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:16.905303 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:17.124542 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:17.124511 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:17.169267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:17.169146 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerStarted","Data":"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d"} Nov 04 05:39:18.104658 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:18.104620 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:39:18.105162 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:18.105136 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:39:18.105290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:18.105195 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:39:19.104185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:19.104152 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:39:19.104702 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:19.104217 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:39:20.104725 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:20.104683 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:39:20.105195 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:20.104741 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:39:20.164018 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:20.163321 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:39:21.104009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:21.103974 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:39:21.104178 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:21.104034 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:39:22.104838 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:22.104804 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:39:22.105329 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:22.104860 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:39:22.495842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:22.495792 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:23.108352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:23.108320 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:39:23.108744 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:23.108683 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:39:23.109465 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:23.109428 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:39:27.438969 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.438933 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:39:27.442475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.442458 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.500465 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.500427 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-fbbad9-serving-cert\"" Nov 04 05:39:27.561898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.561859 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:39:27.576390 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.576356 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.576561 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.576533 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.576613 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.576589 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dvc8\" (UniqueName: \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.576670 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.576655 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.677850 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.677817 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.678094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.677855 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.678094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.677961 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.678094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.677988 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dvc8\" (UniqueName: \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.678094 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:27.678059 5104 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-serving-cert: secret "maas-api-e2e-worker-w6-fbbad9-serving-cert" not found Nov 04 05:39:27.678343 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:27.678157 5104 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls podName:dc9baa3e-5a92-4aaa-a056-12381cc92b9f nodeName:}" failed. No retries permitted until 2026-11-04 05:39:28.178132026 +0000 UTC m=+705.742515307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls") pod "maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" (UID: "dc9baa3e-5a92-4aaa-a056-12381cc92b9f") : secret "maas-api-e2e-worker-w6-fbbad9-serving-cert" not found Nov 04 05:39:27.678698 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.678674 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.680349 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.680320 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.715502 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.715394 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:27.745728 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.745677 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dvc8\" (UniqueName: \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:27.933711 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:27.933673 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:28.140873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:28.140795 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:28.186600 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:28.186568 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:28.189116 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:28.189090 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:28.355390 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:28.355355 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:28.744072 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:28.743999 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:39:28.747763 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:39:28.747731 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9baa3e_5a92_4aaa_a056_12381cc92b9f.slice/crio-59cb16e45ae0bed3698a3190b6710ba941a3aee400f7aeeb5d1c9096b170c564 WatchSource:0}: Error finding container 59cb16e45ae0bed3698a3190b6710ba941a3aee400f7aeeb5d1c9096b170c564: Status 404 returned error can't find the container with id 59cb16e45ae0bed3698a3190b6710ba941a3aee400f7aeeb5d1c9096b170c564 Nov 04 05:39:29.234582 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:29.234546 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" event={"ID":"dc9baa3e-5a92-4aaa-a056-12381cc92b9f","Type":"ContainerStarted","Data":"59cb16e45ae0bed3698a3190b6710ba941a3aee400f7aeeb5d1c9096b170c564"} Nov 04 05:39:30.240776 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:30.240742 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" event={"ID":"dc9baa3e-5a92-4aaa-a056-12381cc92b9f","Type":"ContainerStarted","Data":"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2"} Nov 04 05:39:30.241278 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:30.240845 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:30.372430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:30.372378 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" podStartSLOduration=2.3760454429999998 podStartE2EDuration="3.372360334s" totalImagesPullingTime="996.314891ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:30.370621336 +0000 UTC m=+707.935004636" watchObservedRunningTime="2026-11-04 05:39:30.372360334 +0000 UTC m=+707.936743649" Nov 04 05:39:30.491657 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:39:30.491560 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:39:33.073775 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:33.073703 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:36.253397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:36.253368 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:39:38.272549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:38.272515 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:41.571988 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:41.571958 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:42.209299 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:42.209265 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:42.419457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:42.419396 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:44.138668 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:44.138622 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:47.040860 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:47.040812 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:48.249694 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:48.249644 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:48.931423 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:48.931385 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:49.155714 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:49.155671 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:49.944208 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:49.944168 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:53.741930 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:53.741825 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:54.376291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:54.376248 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:54.592362 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:54.592330 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:55.588856 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:55.588821 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:57.792794 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:57.792756 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:58.421094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:58.421061 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:39:58.631476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:39:58.631438 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:00.491511 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:40:00.491471 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:40:01.258100 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:01.258065 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:04.271153 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:04.271118 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:04.533169 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:04.533092 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:04.894502 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:04.894403 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:05.104275 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:05.104212 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:06.861476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:06.861444 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:08.467421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:08.467388 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:09.157581 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:09.157536 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:09.380069 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:09.380041 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:12.554880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:12.554833 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:14.924709 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:14.924671 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:17.766127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:17.765962 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:18.676033 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:18.675992 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:20.108794 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.108754 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:40:20.112576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.112554 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.170755 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.170726 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"openshift-service-ca.crt\"" Nov 04 05:40:20.226041 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.226012 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"kube-root-ca.crt\"" Nov 04 05:40:20.226200 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.226166 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"default-dockercfg-mzgh6\"" Nov 04 05:40:20.226200 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.226166 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"simulator-w2-961485-kserve-self-signed-certs\"" Nov 04 05:40:20.228334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.228316 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:40:20.268407 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.268375 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:20.279677 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.279646 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.279843 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.279709 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.279843 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.279812 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.279936 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.279918 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.279977 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.279959 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxcn7\" (UniqueName: \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.280037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.280022 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381555 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381470 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381555 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381537 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381562 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxcn7\" (UniqueName: \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381578 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381605 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.381779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381639 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.382002 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.381944 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.382058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.382007 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.382058 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.382031 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.383889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.383863 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.384036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.383990 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.448869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.448836 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxcn7\" (UniqueName: \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\") pod \"simulator-w2-961485-kserve-5dd8c84d6-2vpnk\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:20.724295 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:20.724267 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:21.108000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:21.107889 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:40:21.110734 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:40:21.110705 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee5adc13_1070_4b55_aca9_8dcdac50f754.slice/crio-3b2133947558efb7e562438a2e3a3fe977e59ca3cb8807f0d790ae8f4f76ae2a WatchSource:0}: Error finding container 3b2133947558efb7e562438a2e3a3fe977e59ca3cb8807f0d790ae8f4f76ae2a: Status 404 returned error can't find the container with id 3b2133947558efb7e562438a2e3a3fe977e59ca3cb8807f0d790ae8f4f76ae2a Nov 04 05:40:21.532188 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:21.532155 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" event={"ID":"ee5adc13-1070-4b55-aca9-8dcdac50f754","Type":"ContainerStarted","Data":"3b2133947558efb7e562438a2e3a3fe977e59ca3cb8807f0d790ae8f4f76ae2a"} Nov 04 05:40:22.821561 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:22.821474 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:23.373182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.373147 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:23.564949 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.564902 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" event={"ID":"ee5adc13-1070-4b55-aca9-8dcdac50f754","Type":"ContainerStarted","Data":"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371"} Nov 04 05:40:23.586059 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.586029 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:23.695075 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.694970 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" podStartSLOduration=1.838987572 podStartE2EDuration="3.694953703s" totalImagesPullingTime="1.855966131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:23.692524235 +0000 UTC m=+761.256907546" watchObservedRunningTime="2026-11-04 05:40:23.694953703 +0000 UTC m=+761.259337001" Nov 04 05:40:23.724708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.724677 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:23.798087 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:23.798040 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:28.671955 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:28.671925 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:29.560007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:29.559978 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:30.490675 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:40:30.490641 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:40:30.724699 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:30.724662 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:30.736933 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:30.736906 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:31.616368 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:31.616336 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:40:33.964510 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:33.964478 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:38.897641 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:38.897599 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:39.266398 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:39.266360 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:44.469124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:44.469084 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:49.203703 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:49.203657 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:49.867939 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:49.867908 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:40:55.165970 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:40:55.165935 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:00.115769 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.115736 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:41:00.122842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.122823 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.182668 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.182634 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"openshift-service-ca.crt\"" Nov 04 05:41:00.237978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.237948 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"default-dockercfg-dvnsx\"" Nov 04 05:41:00.238132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.237993 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"premium-w3-a84d48-kserve-self-signed-certs\"" Nov 04 05:41:00.238132 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.238005 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"kube-root-ca.crt\"" Nov 04 05:41:00.240467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.240444 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:41:00.251777 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.251755 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.251894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.251803 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.251980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.251965 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.252041 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.252006 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg9lm\" (UniqueName: \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.252084 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.252036 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.252145 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.252128 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.271364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.271333 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:00.352899 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.352865 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.352929 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.352998 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353037 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg9lm\" (UniqueName: \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353095 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353064 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353358 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353107 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353418 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353400 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353477 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353430 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.353543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.353492 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.355187 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.355159 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.355662 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.355641 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.420402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.420331 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg9lm\" (UniqueName: \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\") pod \"premium-w3-a84d48-kserve-64dfb7bcd4-98pb9\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.435000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.434975 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:00.485976 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.485921 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:00.494090 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:41:00.494050 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:41:00.833334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:00.833306 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:41:00.836272 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:41:00.836203 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod663de138_eb46_4baa_9885_eb8f3bf91544.slice/crio-c91d5c8f070e12cb762aa7aeb3ffe398703eef03e824637f12b265c0e2f58298 WatchSource:0}: Error finding container c91d5c8f070e12cb762aa7aeb3ffe398703eef03e824637f12b265c0e2f58298: Status 404 returned error can't find the container with id c91d5c8f070e12cb762aa7aeb3ffe398703eef03e824637f12b265c0e2f58298 Nov 04 05:41:01.771962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:01.771922 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" event={"ID":"663de138-eb46-4baa-9885-eb8f3bf91544","Type":"ContainerStarted","Data":"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca"} Nov 04 05:41:01.772402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:01.771971 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" event={"ID":"663de138-eb46-4baa-9885-eb8f3bf91544","Type":"ContainerStarted","Data":"c91d5c8f070e12cb762aa7aeb3ffe398703eef03e824637f12b265c0e2f58298"} Nov 04 05:41:01.900064 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:01.900005 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" podStartSLOduration=1.899988553 podStartE2EDuration="1.899988553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:01.899613397 +0000 UTC m=+799.463996697" watchObservedRunningTime="2026-11-04 05:41:01.899988553 +0000 UTC m=+799.464371852" Nov 04 05:41:02.435451 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:02.435416 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:04.911915 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:04.911874 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:05.660185 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:05.660149 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:07.197674 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.197640 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:41:07.201594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.201572 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.315424 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.315396 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"kube-root-ca.crt\"" Nov 04 05:41:07.315685 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.315664 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"openshift-service-ca.crt\"" Nov 04 05:41:07.315788 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.315715 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"simulator-w6-fbbad9-kserve-self-signed-certs\"" Nov 04 05:41:07.315788 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.315721 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fbbad9\"/\"default-dockercfg-prgbv\"" Nov 04 05:41:07.316035 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316015 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.316076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316055 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.316141 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316130 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.316180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316149 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxb67\" (UniqueName: \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.316247 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316178 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.316288 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.316262 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.318094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.318077 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:41:07.358440 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.358390 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:07.416988 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.416955 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417005 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417030 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417054 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxb67\" (UniqueName: \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417093 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417121 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417522 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417609 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417544 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.417609 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.417586 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.419519 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.419462 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.419809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.419793 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.484446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.484412 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxb67\" (UniqueName: \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\") pod \"simulator-w6-fbbad9-kserve-5878cd987f-jxst5\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.514430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.514395 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:07.916812 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:07.916765 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:41:07.918972 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:41:07.918944 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75fb0292_ea10_4e35_a3f2_958d81b65b82.slice/crio-ba2e2d10b526738aae0ed85d51086cc36ceb4a760934e4f1b6eac445727bc03c WatchSource:0}: Error finding container ba2e2d10b526738aae0ed85d51086cc36ceb4a760934e4f1b6eac445727bc03c: Status 404 returned error can't find the container with id ba2e2d10b526738aae0ed85d51086cc36ceb4a760934e4f1b6eac445727bc03c Nov 04 05:41:08.809114 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:08.809068 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" event={"ID":"75fb0292-ea10-4e35-a3f2-958d81b65b82","Type":"ContainerStarted","Data":"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd"} Nov 04 05:41:08.809114 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:08.809121 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" event={"ID":"75fb0292-ea10-4e35-a3f2-958d81b65b82","Type":"ContainerStarted","Data":"ba2e2d10b526738aae0ed85d51086cc36ceb4a760934e4f1b6eac445727bc03c"} Nov 04 05:41:08.938367 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:08.938319 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" podStartSLOduration=1.938305699 podStartE2EDuration="1.938305699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:08.937391998 +0000 UTC m=+806.501775297" watchObservedRunningTime="2026-11-04 05:41:08.938305699 +0000 UTC m=+806.502688998" Nov 04 05:41:09.514933 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:09.514896 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:10.435528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:10.435492 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:10.448393 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:10.448368 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:10.830936 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:10.830907 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:41:10.969646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:10.969596 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:13.837010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:13.836968 5104 generic.go:356] "Generic (PLEG): container finished" podID="48ae5a00-e13e-4f71-ab93-07b8d6fa5716" containerID="a0430ef793a01554b05a36de5ff9ac48661c666c7801f6ff1952da7410cd21a8" exitCode=1 Nov 04 05:41:13.837482 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:13.837036 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" event={"ID":"48ae5a00-e13e-4f71-ab93-07b8d6fa5716","Type":"ContainerDied","Data":"a0430ef793a01554b05a36de5ff9ac48661c666c7801f6ff1952da7410cd21a8"} Nov 04 05:41:13.837482 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:13.837424 5104 scope.go:122] "RemoveContainer" containerID="a0430ef793a01554b05a36de5ff9ac48661c666c7801f6ff1952da7410cd21a8" Nov 04 05:41:14.845065 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:14.845030 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" event={"ID":"48ae5a00-e13e-4f71-ab93-07b8d6fa5716","Type":"ContainerStarted","Data":"6db43d2715e30b728974e031c816f2c19f86ce8d32a89c2b95fb705fe496ef63"} Nov 04 05:41:14.845569 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:14.845381 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:41:16.189619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:16.189582 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:17.515529 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:17.515498 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:17.527650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:17.527632 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:17.874036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:17.873961 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:41:21.493748 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:21.493569 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:25.853623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:25.853594 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-mlw5m" Nov 04 05:41:26.756612 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:26.756580 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:30.491617 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:41:30.491581 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:41:31.967560 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:31.967513 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:35.449874 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.449839 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:41:35.453862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.453840 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.567576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.567537 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"kube-root-ca.crt\"" Nov 04 05:41:35.567576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.567543 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"openshift-service-ca.crt\"" Nov 04 05:41:35.567798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.567662 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"default-dockercfg-8ts28\"" Nov 04 05:41:35.567798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.567688 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"premium-w4-901a00-kserve-self-signed-certs\"" Nov 04 05:41:35.570107 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.570089 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:41:35.578150 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578122 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w77c\" (UniqueName: \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.578299 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578168 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.578299 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578193 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.578421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578377 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.578483 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578434 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.578539 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.578489 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.619732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.619628 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:35.679713 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.679681 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w77c\" (UniqueName: \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.679900 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.679732 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.679900 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.679803 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.679900 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.679873 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.680091 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.679942 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.680091 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.680015 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.680196 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.680117 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.680196 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.680156 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.680402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.680382 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.682345 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.682318 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.682711 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.682691 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.751156 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.751110 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w77c\" (UniqueName: \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\") pod \"premium-w4-901a00-kserve-d68c67695-4pbct\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:35.765773 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:35.765746 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:36.161013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:36.160988 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:41:36.163015 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:41:36.162989 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88fca7cd_4489_4e29_9d24_e3d7b38db4f3.slice/crio-4a0fcc3ab740ff2d5d8c0e3181e5bf0f22d2a08141e4002cbcc9ec34d71c4353 WatchSource:0}: Error finding container 4a0fcc3ab740ff2d5d8c0e3181e5bf0f22d2a08141e4002cbcc9ec34d71c4353: Status 404 returned error can't find the container with id 4a0fcc3ab740ff2d5d8c0e3181e5bf0f22d2a08141e4002cbcc9ec34d71c4353 Nov 04 05:41:36.967213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:36.967176 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" event={"ID":"88fca7cd-4489-4e29-9d24-e3d7b38db4f3","Type":"ContainerStarted","Data":"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407"} Nov 04 05:41:36.967613 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:36.967242 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" event={"ID":"88fca7cd-4489-4e29-9d24-e3d7b38db4f3","Type":"ContainerStarted","Data":"4a0fcc3ab740ff2d5d8c0e3181e5bf0f22d2a08141e4002cbcc9ec34d71c4353"} Nov 04 05:41:37.096466 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:37.096412 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" podStartSLOduration=2.096396872 podStartE2EDuration="2.096396872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:37.094860989 +0000 UTC m=+834.659244291" watchObservedRunningTime="2026-11-04 05:41:37.096396872 +0000 UTC m=+834.660780176" Nov 04 05:41:37.282423 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:37.282314 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:37.765914 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:37.765881 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:41.548578 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:41.548536 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:42.599186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:42.599151 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:45.766648 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:45.766612 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:45.781414 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:45.781382 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:46.035140 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:46.035056 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:41:47.886470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:47.886426 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:50.546996 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:50.546963 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:53.057072 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:53.057040 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:56.782574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.782538 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:41:56.786311 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.786282 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886615 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886583 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886615 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886617 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886831 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886756 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64xb\" (UniqueName: \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886831 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886791 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886831 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886814 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.886943 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.886839 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.904509 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.904485 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"premium-w2-961485-kserve-self-signed-certs\"" Nov 04 05:41:56.906839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.906817 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:41:56.988124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988077 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988434 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988471 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988600 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64xb\" (UniqueName: \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988656 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988694 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.988929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.988687 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.989324 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.989051 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.989324 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.989185 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.991597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.991574 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:56.994840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:56.994810 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:57.032370 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:57.032341 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:41:57.055693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:57.055630 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64xb\" (UniqueName: \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\") pod \"premium-w2-961485-kserve-7d9db758dc-4h256\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:57.099082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:57.099054 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:57.489292 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:57.489269 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:41:57.491548 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:41:57.491522 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c40d787_40fe_4cd2_aa5d_b454ea9b961f.slice/crio-c2b6901fa7001489e4604585edf24039163ab4d956466bac121030f043d8b9ac WatchSource:0}: Error finding container c2b6901fa7001489e4604585edf24039163ab4d956466bac121030f043d8b9ac: Status 404 returned error can't find the container with id c2b6901fa7001489e4604585edf24039163ab4d956466bac121030f043d8b9ac Nov 04 05:41:58.092842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:58.092802 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" event={"ID":"1c40d787-40fe-4cd2-aa5d-b454ea9b961f","Type":"ContainerStarted","Data":"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d"} Nov 04 05:41:58.093241 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:58.092847 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" event={"ID":"1c40d787-40fe-4cd2-aa5d-b454ea9b961f","Type":"ContainerStarted","Data":"c2b6901fa7001489e4604585edf24039163ab4d956466bac121030f043d8b9ac"} Nov 04 05:41:58.100240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:58.100200 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:41:58.227705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:58.227658 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" podStartSLOduration=2.227643386 podStartE2EDuration="2.227643386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:41:58.226363608 +0000 UTC m=+855.790746907" watchObservedRunningTime="2026-11-04 05:41:58.227643386 +0000 UTC m=+855.792026684" Nov 04 05:41:58.273952 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:41:58.273920 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:00.491252 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:42:00.491204 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:42:01.493804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:01.493767 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:01.879749 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:01.879670 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:02.088123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:02.088093 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:03.572171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:03.572136 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:05.295872 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:05.295822 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:05.506475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:05.506441 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:07.100070 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:07.100031 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:42:07.113128 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:07.113105 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:42:07.154710 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:07.154686 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:42:08.570300 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.570266 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:42:08.574199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.574177 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.653213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.653184 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:42:08.666897 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.666868 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:08.688563 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.688539 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"openshift-service-ca.crt\"" Nov 04 05:42:08.688715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.688627 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"kube-root-ca.crt\"" Nov 04 05:42:08.688715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.688690 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"default-dockercfg-cqcz7\"" Nov 04 05:42:08.694339 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.694320 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"premium-w1-098178-kserve-self-signed-certs\"" Nov 04 05:42:08.704437 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704412 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.704564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704462 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.704564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704527 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5cqv\" (UniqueName: \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.704564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704555 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.704667 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704593 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.704708 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.704675 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805154 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805116 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805164 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5cqv\" (UniqueName: \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805199 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805258 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805318 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805611 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805396 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805671 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805604 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805671 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805657 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.805873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.805850 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.810460 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.810439 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.810889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.810865 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.871845 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.871771 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5cqv\" (UniqueName: \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\") pod \"premium-w1-098178-kserve-6d97ffb97b-rk6ts\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:08.875434 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.875397 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:08.887240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:08.887204 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:09.283365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:09.283341 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:42:09.285297 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:42:09.285266 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca923b6c_2a90_4bc5_a9c0_03a04dc36baf.slice/crio-b63f0d7e146e9dc40ebe71494d2d515465bc7462c21e550c6211580d6d408327 WatchSource:0}: Error finding container b63f0d7e146e9dc40ebe71494d2d515465bc7462c21e550c6211580d6d408327: Status 404 returned error can't find the container with id b63f0d7e146e9dc40ebe71494d2d515465bc7462c21e550c6211580d6d408327 Nov 04 05:42:10.165134 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:10.165091 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" event={"ID":"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf","Type":"ContainerStarted","Data":"d891d0e09cc5d387a5302aec5922447cd46ec1cca6afab889f094eace3004c64"} Nov 04 05:42:10.165665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:10.165148 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" event={"ID":"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf","Type":"ContainerStarted","Data":"b63f0d7e146e9dc40ebe71494d2d515465bc7462c21e550c6211580d6d408327"} Nov 04 05:42:10.296615 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:10.296558 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" podStartSLOduration=2.296545025 podStartE2EDuration="2.296545025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:10.293525725 +0000 UTC m=+867.857909025" watchObservedRunningTime="2026-11-04 05:42:10.296545025 +0000 UTC m=+867.860928303" Nov 04 05:42:10.887459 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:10.887425 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:14.046155 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.046120 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:42:14.049907 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.049886 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.062179 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.062153 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:14.124351 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.124321 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"openshift-service-ca.crt\"" Nov 04 05:42:14.152265 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152214 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.152398 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152361 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.152398 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152390 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ln8d\" (UniqueName: \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.152541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152515 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.152578 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152562 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.152679 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.152666 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.163515 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.163489 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"kube-root-ca.crt\"" Nov 04 05:42:14.163650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.163608 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"premium-w5-283815-kserve-self-signed-certs\"" Nov 04 05:42:14.163650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.163621 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-283815\"/\"default-dockercfg-jdn6x\"" Nov 04 05:42:14.166103 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.166081 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:42:14.253780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253739 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.253984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253830 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.253984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253867 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ln8d\" (UniqueName: \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.253984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253908 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.253984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253935 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.254196 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.253985 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.254334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.254311 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.254411 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.254327 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.254467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.254405 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.255974 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.255957 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.256459 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.256438 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.281031 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.281000 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:14.321123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.321058 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ln8d\" (UniqueName: \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\") pod \"premium-w5-283815-kserve-6689f74d4d-j9xfd\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.364369 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.364339 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:14.758415 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:14.758263 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:42:14.760916 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:42:14.760879 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1606b823_fa2d_45a8_bc5b_d43064b2250d.slice/crio-53882bf56504c367cffc59173622470efc1a73dc6432a4215d50eba489a47bca WatchSource:0}: Error finding container 53882bf56504c367cffc59173622470efc1a73dc6432a4215d50eba489a47bca: Status 404 returned error can't find the container with id 53882bf56504c367cffc59173622470efc1a73dc6432a4215d50eba489a47bca Nov 04 05:42:15.196759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:15.196673 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" event={"ID":"1606b823-fa2d-45a8-bc5b-d43064b2250d","Type":"ContainerStarted","Data":"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060"} Nov 04 05:42:15.196759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:15.196721 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" event={"ID":"1606b823-fa2d-45a8-bc5b-d43064b2250d","Type":"ContainerStarted","Data":"53882bf56504c367cffc59173622470efc1a73dc6432a4215d50eba489a47bca"} Nov 04 05:42:15.326418 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:15.326368 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" podStartSLOduration=2.326354368 podStartE2EDuration="2.326354368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:15.32499899 +0000 UTC m=+872.889382288" watchObservedRunningTime="2026-11-04 05:42:15.326354368 +0000 UTC m=+872.890737667" Nov 04 05:42:15.364558 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:15.364523 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:18.888089 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:18.888051 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:18.901294 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:18.901261 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:19.254988 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:19.254957 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:42:19.363822 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:19.363786 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:22.533186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:22.533138 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:24.365474 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:24.365440 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:24.378304 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:24.378280 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:24.647818 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:24.647724 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:25.272857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:25.272829 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:42:29.857933 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:29.857901 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:30.491201 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:42:30.491166 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:42:35.082400 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:35.082361 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:40.468824 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:40.468792 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:41.782210 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:41.782173 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:45.828604 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:45.828568 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:51.077673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:51.077579 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:42:56.381644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:42:56.381602 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:00.490845 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:43:00.490810 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:43:01.760737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:01.760704 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:05.674882 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:05.674846 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:06.975780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:06.975740 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:12.270808 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:12.270774 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:17.711883 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:17.711829 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:23.215602 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:23.215559 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:28.581790 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:28.581746 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:30.498282 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:43:30.498244 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:43:32.575475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:32.575444 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:33.977630 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:33.977594 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:39.314275 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:39.314199 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:44.776583 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:44.776548 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:50.274356 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:50.274326 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:43:55.471908 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:43:55.471865 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:00.490433 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:00.490390 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:44:00.785762 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:00.785683 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:05.682241 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.682183 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7212-743d88"] Nov 04 05:44:05.687841 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.687817 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:05.748210 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.748177 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7212-743d88"] Nov 04 05:44:05.805878 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.805846 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lxtlf\"" Nov 04 05:44:05.839280 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.839245 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tnms\" (UniqueName: \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\") pod \"test-curl-7212-743d88\" (UID: \"391b4bb6-af84-4050-8354-a6b7d97cdae9\") " pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:05.945020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:05.944891 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tnms\" (UniqueName: \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\") pod \"test-curl-7212-743d88\" (UID: \"391b4bb6-af84-4050-8354-a6b7d97cdae9\") " pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:06.024174 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.024142 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tnms\" (UniqueName: \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\") pod \"test-curl-7212-743d88\" (UID: \"391b4bb6-af84-4050-8354-a6b7d97cdae9\") " pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:06.099255 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.099206 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:06.301984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.301950 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:06.316309 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.316276 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:06.717541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.717512 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7212-743d88"] Nov 04 05:44:06.734171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.734145 5104 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:44:06.898652 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:06.898617 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7212-743d88" event={"ID":"391b4bb6-af84-4050-8354-a6b7d97cdae9","Type":"ContainerStarted","Data":"083c1604d2658e3b702942f9f28916cfa3836a7f40f8ffcd05b70e95827b7964"} Nov 04 05:44:10.927661 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:10.927620 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7212-743d88" event={"ID":"391b4bb6-af84-4050-8354-a6b7d97cdae9","Type":"ContainerStarted","Data":"7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70"} Nov 04 05:44:10.937261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:10.937188 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:11.053470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:11.053415 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7212-743d88" podStartSLOduration=2.454505462 podStartE2EDuration="6.053400487s" totalImagesPullingTime="3.598895025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:11.052003342 +0000 UTC m=+988.616386642" watchObservedRunningTime="2026-11-04 05:44:11.053400487 +0000 UTC m=+988.617783786" Nov 04 05:44:11.147044 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:11.147008 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:11.505080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:11.505009 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:11.823652 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:11.823556 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:12.744086 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:12.744050 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7212-743d88"] Nov 04 05:44:12.747414 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:12.747386 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7212-743d88"] Nov 04 05:44:12.939190 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:12.939151 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7212-743d88" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" containerName="test-curl-7212-743d88" containerID="cri-o://7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70" gracePeriod=2 Nov 04 05:44:12.996270 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:12.996167 5104 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7212-743d88\" is forbidden: User \"system:node:ip-10-0-139-34.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-34.us-west-2.compute.internal' and this object" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:13.037644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:13.037606 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:15.195925 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.195869 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:15.207963 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.207927 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:15.355575 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.355451 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\" (UniqueName: \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\") pod \"391b4bb6-af84-4050-8354-a6b7d97cdae9\" (UID: \"391b4bb6-af84-4050-8354-a6b7d97cdae9\") " Nov 04 05:44:15.365150 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.357504 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms" pod "391b4bb6-af84-4050-8354-a6b7d97cdae9" (UID: "391b4bb6-af84-4050-8354-a6b7d97cdae9"). InnerVolumeSpecName "kube-api-access-7tnms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:15.456315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.456240 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tnms\" (UniqueName: \"kubernetes.io/projected/391b4bb6-af84-4050-8354-a6b7d97cdae9-kube-api-access-7tnms\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:15.915101 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.915067 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:44:15.962183 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.962142 5104 generic.go:356] "Generic (PLEG): container finished" podID="391b4bb6-af84-4050-8354-a6b7d97cdae9" containerID="7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70" exitCode=137 Nov 04 05:44:15.962382 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.962196 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:15.962382 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.962251 5104 scope.go:122] "RemoveContainer" containerID="7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70" Nov 04 05:44:15.972615 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.972594 5104 scope.go:122] "RemoveContainer" containerID="7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70" Nov 04 05:44:15.972925 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:15.972902 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70\": container with ID starting with 7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70 not found: ID does not exist" containerID="7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70" Nov 04 05:44:15.973004 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:15.972939 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70"} err="failed to get container status \"7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70\": rpc error: code = NotFound desc = could not find container \"7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70\": container with ID starting with 7b51119dd2b05d6fccf54dcd4308645954b8166f5e3ffeb2018d3a3cb9ea0e70 not found: ID does not exist" Nov 04 05:44:16.100604 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:16.100565 5104 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7212-743d88\" is forbidden: User \"system:node:ip-10-0-139-34.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-34.us-west-2.compute.internal' and this object" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" pod="opendatahub/test-curl-7212-743d88" Nov 04 05:44:16.131720 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:16.131686 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:16.551467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:16.551431 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" path="/var/lib/kubelet/pods/391b4bb6-af84-4050-8354-a6b7d97cdae9/volumes" Nov 04 05:44:16.823422 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:16.823339 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:17.705532 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:17.705452 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:18.012563 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:18.012533 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:44:18.196000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:18.195970 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:18.404837 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:18.404753 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:18.616442 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:18.616406 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:19.550041 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.550010 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt"] Nov 04 05:44:19.550521 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.550458 5104 cpu_manager.go:414] "containerMap: removing container" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" containerName="test-curl-7212-743d88" Nov 04 05:44:19.550521 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.550473 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" containerName="test-curl-7212-743d88" Nov 04 05:44:19.550635 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.550578 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="391b4bb6-af84-4050-8354-a6b7d97cdae9" containerName="test-curl-7212-743d88" Nov 04 05:44:19.555335 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.555310 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.615446 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.615414 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:44:19.670991 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.670957 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:44:19.673404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.673383 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt"] Nov 04 05:44:19.696448 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.696417 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb96l\" (UniqueName: \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.696567 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.696546 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.696657 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.696644 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.797138 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.797103 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.797283 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.797143 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb96l\" (UniqueName: \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.797398 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.797365 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.797975 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.797952 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.799567 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.799543 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.864302 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.864234 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb96l\" (UniqueName: \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\") pod \"maas-api-revoke-sub-10107d08-q5zzt\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:19.868008 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:19.867930 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:20.252695 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:20.252669 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt"] Nov 04 05:44:20.255361 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:44:20.255333 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be6daf6_f6b1_40c8_8dfc_1ce90443af6f.slice/crio-2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047 WatchSource:0}: Error finding container 2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047: Status 404 returned error can't find the container with id 2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047 Nov 04 05:44:20.330313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:20.330277 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:20.996940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:20.996849 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" event={"ID":"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f","Type":"ContainerStarted","Data":"2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047"} Nov 04 05:44:22.116135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:22.116104 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:22.383267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:22.383154 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:23.447096 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:23.447059 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:44:23.660675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:23.660641 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:24.350731 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:24.350694 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:27.451404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:27.451368 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:28.467871 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:28.467834 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:29.044456 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:29.044416 5104 generic.go:356] "Generic (PLEG): container finished" podID="1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" containerID="5f554f468c85922c6375ab7cc6d18e54482994b5852d02b88881112b6250db0a" exitCode=0 Nov 04 05:44:29.044622 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:29.044495 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" event={"ID":"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f","Type":"ContainerDied","Data":"5f554f468c85922c6375ab7cc6d18e54482994b5852d02b88881112b6250db0a"} Nov 04 05:44:30.198173 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.197861 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:30.311169 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.311080 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\" (UniqueName: \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\") pod \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " Nov 04 05:44:30.311384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.311257 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\") pod \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " Nov 04 05:44:30.311461 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.311444 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\") pod \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\" (UID: \"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f\") " Nov 04 05:44:30.311710 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.311679 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca" pod "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" (UID: "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:30.311887 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.311865 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-openshift-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:30.316676 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.316643 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca" pod "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" (UID: "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:30.316779 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.316670 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l" pod "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" (UID: "1be6daf6-f6b1-40c8-8dfc-1ce90443af6f"). InnerVolumeSpecName "kube-api-access-wb96l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:30.413431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.413393 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb96l\" (UniqueName: \"kubernetes.io/projected/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-kube-api-access-wb96l\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:30.413431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.413427 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f-maas-api-xks-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:30.445098 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.445060 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:30.491864 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:30.491827 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:44:30.940270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:30.940129 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerName="main" containerID="cri-o://d891d0e09cc5d387a5302aec5922447cd46ec1cca6afab889f094eace3004c64" gracePeriod=45 Nov 04 05:44:31.068839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.068088 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" event={"ID":"1be6daf6-f6b1-40c8-8dfc-1ce90443af6f","Type":"ContainerDied","Data":"2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047"} Nov 04 05:44:31.068839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.068149 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a6cf7bf9676956c9e9463a013d8ce1a105a0343746f800009cc8698b6066047" Nov 04 05:44:31.068839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.068098 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt" Nov 04 05:44:31.069897 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.069872 5104 generic.go:356] "Generic (PLEG): container finished" podID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerID="d891d0e09cc5d387a5302aec5922447cd46ec1cca6afab889f094eace3004c64" exitCode=0 Nov 04 05:44:31.070025 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.069959 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" event={"ID":"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf","Type":"ContainerDied","Data":"d891d0e09cc5d387a5302aec5922447cd46ec1cca6afab889f094eace3004c64"} Nov 04 05:44:31.208888 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.208727 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:31.232087 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.232066 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:44:31.321816 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.321778 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.321993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.321832 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.321993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.321861 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\" (UniqueName: \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.321993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.321945 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.322152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.321998 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\" (UniqueName: \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.322152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322058 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\") pod \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\" (UID: \"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf\") " Nov 04 05:44:31.322152 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322069 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.322348 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322201 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.322479 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322446 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.322564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322506 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.322564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.322523 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.324501 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.324475 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.324597 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.324521 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "kube-api-access-k5cqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:31.324697 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.324678 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs" pod "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" (UID: "ca923b6c-2a90-4bc5-a9c0-03a04dc36baf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:31.423852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.423819 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.423852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.423849 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.423852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.423858 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.424090 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:31.423868 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5cqv\" (UniqueName: \"kubernetes.io/projected/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf-kube-api-access-k5cqv\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:32.088263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.085755 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:32.088263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.087253 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" event={"ID":"ca923b6c-2a90-4bc5-a9c0-03a04dc36baf","Type":"ContainerDied","Data":"b63f0d7e146e9dc40ebe71494d2d515465bc7462c21e550c6211580d6d408327"} Nov 04 05:44:32.088263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.087280 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" Nov 04 05:44:32.088263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.087306 5104 scope.go:122] "RemoveContainer" containerID="d891d0e09cc5d387a5302aec5922447cd46ec1cca6afab889f094eace3004c64" Nov 04 05:44:32.226929 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.226889 5104 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded" Nov 04 05:44:32.270778 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.270744 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:44:32.301439 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.301414 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:32.324938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.324909 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-098178/premium-w1-098178-kserve-6d97ffb97b-rk6ts"] Nov 04 05:44:32.549454 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:32.549427 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" path="/var/lib/kubelet/pods/ca923b6c-2a90-4bc5-a9c0-03a04dc36baf/volumes" Nov 04 05:44:33.032976 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.032938 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerName="main" containerID="cri-o://bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060" gracePeriod=45 Nov 04 05:44:33.111747 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.111721 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:33.166242 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166147 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:44:33.166800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166776 5104 cpu_manager.go:414] "containerMap: removing container" podUID="1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" containerName="revoke-keys" Nov 04 05:44:33.166800 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166801 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" containerName="revoke-keys" Nov 04 05:44:33.167115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166870 5104 cpu_manager.go:414] "containerMap: removing container" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerName="main" Nov 04 05:44:33.167115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166879 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerName="main" Nov 04 05:44:33.167115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166976 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" containerName="revoke-keys" Nov 04 05:44:33.167115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.166993 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca923b6c-2a90-4bc5-a9c0-03a04dc36baf" containerName="main" Nov 04 05:44:33.177872 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.177017 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.241593 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242483 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rw99\" (UniqueName: \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242532 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242576 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242622 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242661 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.243397 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.242732 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.276330 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.276301 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"distinct-w4-901a00-kserve-self-signed-certs\"" Nov 04 05:44:33.322827 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.322746 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:33.343432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343403 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343502 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rw99\" (UniqueName: \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343546 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343582 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343694 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343622 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343694 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343653 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343923 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343901 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.343999 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.343961 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.344354 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.344331 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.346314 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.346281 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.346411 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.346337 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.350331 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.350309 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:44:33.410945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.410913 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rw99\" (UniqueName: \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\") pod \"distinct-w4-901a00-kserve-5bb5ffc674-qxrgj\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.444996 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.444965 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\" (UniqueName: \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445158 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445026 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445264 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445209 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\" (UniqueName: \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445320 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445282 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.445394 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445344 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445394 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445377 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445504 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445428 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\") pod \"1606b823-fa2d-45a8-bc5b-d43064b2250d\" (UID: \"1606b823-fa2d-45a8-bc5b-d43064b2250d\") " Nov 04 05:44:33.445972 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.445778 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.446165 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.446118 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.446165 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.446131 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.447841 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.447788 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "kube-api-access-7ln8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:33.447949 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.447866 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:33.448171 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.448152 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm" pod "1606b823-fa2d-45a8-bc5b-d43064b2250d" (UID: "1606b823-fa2d-45a8-bc5b-d43064b2250d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:33.496796 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.496758 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:33.546684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.546566 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1606b823-fa2d-45a8-bc5b-d43064b2250d-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.546684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.546606 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.546684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.546619 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.546684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.546634 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1606b823-fa2d-45a8-bc5b-d43064b2250d-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.546684 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.546652 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ln8d\" (UniqueName: \"kubernetes.io/projected/1606b823-fa2d-45a8-bc5b-d43064b2250d-kube-api-access-7ln8d\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.904951 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:33.904918 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:44:33.905832 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:44:33.905795 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddecfcd7c_7e35_4b21_b096_fad337c354ca.slice/crio-ba9f02e29d7c99f5f377b8826c9af970d86ca1261113072be0c0262a1ebba3a3 WatchSource:0}: Error finding container ba9f02e29d7c99f5f377b8826c9af970d86ca1261113072be0c0262a1ebba3a3: Status 404 returned error can't find the container with id ba9f02e29d7c99f5f377b8826c9af970d86ca1261113072be0c0262a1ebba3a3 Nov 04 05:44:34.102026 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.101989 5104 generic.go:356] "Generic (PLEG): container finished" podID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerID="bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060" exitCode=0 Nov 04 05:44:34.102180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.102034 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" Nov 04 05:44:34.102180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.102076 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" event={"ID":"1606b823-fa2d-45a8-bc5b-d43064b2250d","Type":"ContainerDied","Data":"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060"} Nov 04 05:44:34.102180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.102124 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" event={"ID":"1606b823-fa2d-45a8-bc5b-d43064b2250d","Type":"ContainerDied","Data":"53882bf56504c367cffc59173622470efc1a73dc6432a4215d50eba489a47bca"} Nov 04 05:44:34.102180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.102146 5104 scope.go:122] "RemoveContainer" containerID="bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060" Nov 04 05:44:34.104017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.103977 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" event={"ID":"decfcd7c-7e35-4b21-b096-fad337c354ca","Type":"ContainerStarted","Data":"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0"} Nov 04 05:44:34.104127 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.104024 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" event={"ID":"decfcd7c-7e35-4b21-b096-fad337c354ca","Type":"ContainerStarted","Data":"ba9f02e29d7c99f5f377b8826c9af970d86ca1261113072be0c0262a1ebba3a3"} Nov 04 05:44:34.111825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.111807 5104 scope.go:122] "RemoveContainer" containerID="bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060" Nov 04 05:44:34.112082 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:34.112057 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060\": container with ID starting with bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060 not found: ID does not exist" containerID="bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060" Nov 04 05:44:34.112136 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.112090 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060"} err="failed to get container status \"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060\": rpc error: code = NotFound desc = could not find container \"bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060\": container with ID starting with bdf121c4b2100934390c6535f4e908e8c9b98c90c4a1f174c73d27b49f69b060 not found: ID does not exist" Nov 04 05:44:34.262405 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.262365 5104 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded" Nov 04 05:44:34.283313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.283280 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:44:34.289514 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.289488 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-283815/premium-w5-283815-kserve-6689f74d4d-j9xfd"] Nov 04 05:44:34.416046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.415988 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" podStartSLOduration=1.415974179 podStartE2EDuration="1.415974179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:34.41421078 +0000 UTC m=+1011.978594080" watchObservedRunningTime="2026-11-04 05:44:34.415974179 +0000 UTC m=+1011.980357519" Nov 04 05:44:34.497909 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.497873 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:34.553419 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.553330 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" path="/var/lib/kubelet/pods/1606b823-fa2d-45a8-bc5b-d43064b2250d/volumes" Nov 04 05:44:34.623087 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:34.623053 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:35.111083 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:35.111039 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:35.844565 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:35.844523 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:36.049464 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:36.049426 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:36.492472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:36.492443 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:36.959637 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:36.959544 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:37.168115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:37.168079 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:37.716334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:37.716303 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:38.467193 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.467155 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" podUID="75fb0292-ea10-4e35-a3f2-958d81b65b82" containerName="main" containerID="cri-o://ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd" gracePeriod=45 Nov 04 05:44:38.677612 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.677580 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:38.729162 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.729105 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:44:38.801287 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801256 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\" (UniqueName: \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801287 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801293 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801312 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\" (UniqueName: \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801405 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801499 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801493 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801537 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\") pod \"75fb0292-ea10-4e35-a3f2-958d81b65b82\" (UID: \"75fb0292-ea10-4e35-a3f2-958d81b65b82\") " Nov 04 05:44:38.801647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801573 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.801849 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801823 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.801944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801854 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.801944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.801886 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.803567 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.803544 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:38.803656 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.803602 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:38.803706 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.803683 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67" pod "75fb0292-ea10-4e35-a3f2-958d81b65b82" (UID: "75fb0292-ea10-4e35-a3f2-958d81b65b82"). InnerVolumeSpecName "kube-api-access-wxb67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:38.903078 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.903047 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.903078 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.903077 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.903078 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.903086 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fb0292-ea10-4e35-a3f2-958d81b65b82-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.903349 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.903096 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fb0292-ea10-4e35-a3f2-958d81b65b82-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:38.903349 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:38.903106 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxb67\" (UniqueName: \"kubernetes.io/projected/75fb0292-ea10-4e35-a3f2-958d81b65b82-kube-api-access-wxb67\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.136352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.136255 5104 generic.go:356] "Generic (PLEG): container finished" podID="75fb0292-ea10-4e35-a3f2-958d81b65b82" containerID="ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd" exitCode=0 Nov 04 05:44:39.136352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.136300 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" Nov 04 05:44:39.136565 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.136343 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" event={"ID":"75fb0292-ea10-4e35-a3f2-958d81b65b82","Type":"ContainerDied","Data":"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd"} Nov 04 05:44:39.136565 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.136388 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5" event={"ID":"75fb0292-ea10-4e35-a3f2-958d81b65b82","Type":"ContainerDied","Data":"ba2e2d10b526738aae0ed85d51086cc36ceb4a760934e4f1b6eac445727bc03c"} Nov 04 05:44:39.136565 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.136405 5104 scope.go:122] "RemoveContainer" containerID="ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd" Nov 04 05:44:39.147149 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.147134 5104 scope.go:122] "RemoveContainer" containerID="ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd" Nov 04 05:44:39.147417 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:39.147384 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd\": container with ID starting with ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd not found: ID does not exist" containerID="ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd" Nov 04 05:44:39.147492 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.147417 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd"} err="failed to get container status \"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd\": rpc error: code = NotFound desc = could not find container \"ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd\": container with ID starting with ef84b6a98ca04b9041b19dfa5f6a8e975334ba08004a8f378ec61fa05c3f79cd not found: ID does not exist" Nov 04 05:44:39.318781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.318750 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:44:39.374245 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:39.374183 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-fbbad9/simulator-w6-fbbad9-kserve-5878cd987f-jxst5"] Nov 04 05:44:40.549837 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:40.549807 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75fb0292-ea10-4e35-a3f2-958d81b65b82" path="/var/lib/kubelet/pods/75fb0292-ea10-4e35-a3f2-958d81b65b82/volumes" Nov 04 05:44:40.646384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:40.646355 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:41.366320 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366289 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:44:41.366748 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366732 5104 cpu_manager.go:414] "containerMap: removing container" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerName="main" Nov 04 05:44:41.366799 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366750 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerName="main" Nov 04 05:44:41.366799 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366768 5104 cpu_manager.go:414] "containerMap: removing container" podUID="75fb0292-ea10-4e35-a3f2-958d81b65b82" containerName="main" Nov 04 05:44:41.366799 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366773 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75fb0292-ea10-4e35-a3f2-958d81b65b82" containerName="main" Nov 04 05:44:41.366894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366850 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="75fb0292-ea10-4e35-a3f2-958d81b65b82" containerName="main" Nov 04 05:44:41.366894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.366863 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="1606b823-fa2d-45a8-bc5b-d43064b2250d" containerName="main" Nov 04 05:44:41.371401 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.371383 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.385181 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.385159 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:41.428516 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428489 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428645 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428527 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428631 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428673 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428785 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428740 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428868 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428851 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.428928 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428894 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.429001 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.428987 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48vq5\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.429068 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.429049 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.430469 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.430451 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-961485-openshift-default-dockercfg-qhhl7\"" Nov 04 05:44:41.487617 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.487581 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:44:41.530240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530193 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530401 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530272 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530401 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530310 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48vq5\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530401 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530347 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530573 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530406 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530573 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530431 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530573 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530477 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530721 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530672 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530742 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530853 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.530955 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530933 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.531009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.530963 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.531165 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.531146 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.531201 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.531169 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.532943 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.532916 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.533037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.532982 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.597691 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.597661 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.598009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.597934 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48vq5\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.685656 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.685576 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:41.736634 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:41.728576 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:42.012666 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:42.012628 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:42.160792 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:42.160660 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:44:42.163260 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:44:42.163217 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66cbeee_0751_4aca_af9d_73e1c7c77901.slice/crio-1da0a015bee315fb932f69c751b244a5ed921f2f7fb683c5dde7d9553fc7116c WatchSource:0}: Error finding container 1da0a015bee315fb932f69c751b244a5ed921f2f7fb683c5dde7d9553fc7116c: Status 404 returned error can't find the container with id 1da0a015bee315fb932f69c751b244a5ed921f2f7fb683c5dde7d9553fc7116c Nov 04 05:44:42.165623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:42.165585 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:44:42.165726 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:42.165671 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:44:42.165767 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:42.165719 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:44:43.127775 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.127637 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:43.166619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.166582 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" event={"ID":"d66cbeee-0751-4aca-af9d-73e1c7c77901","Type":"ContainerStarted","Data":"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf"} Nov 04 05:44:43.166841 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.166637 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" event={"ID":"d66cbeee-0751-4aca-af9d-73e1c7c77901","Type":"ContainerStarted","Data":"1da0a015bee315fb932f69c751b244a5ed921f2f7fb683c5dde7d9553fc7116c"} Nov 04 05:44:43.359203 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.358999 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podStartSLOduration=2.358980047 podStartE2EDuration="2.358980047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:43.358788102 +0000 UTC m=+1020.923171418" watchObservedRunningTime="2026-11-04 05:44:43.358980047 +0000 UTC m=+1020.923363346" Nov 04 05:44:43.497260 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.497200 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:43.510208 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.510178 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:43.686346 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.686314 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:43.688015 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.687986 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:43.688161 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.688038 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:43.988780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:43.988752 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:44.186021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.185985 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:44:44.203261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.203208 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:44.414618 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.414520 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:44.619053 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.619017 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:44.686357 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.686273 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:44.686357 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:44.686329 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:45.686579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:45.686541 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:45.686997 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:45.686607 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:46.687037 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:46.686992 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:46.687460 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:46.687072 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:47.506576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:47.506540 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:47.686497 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:47.686457 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:47.686694 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:47.686539 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:48.214729 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:48.214683 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:48.686401 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:48.686371 5104 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 05:44:48.686576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:48.686422 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 05:44:49.023367 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.023336 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:44:49.023583 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.023551 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" podUID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" containerName="maas-api" containerID="cri-o://f0c899ae555a776151616d7612d58ed7aeaa426aaa6303d509adf0ac351ef001" gracePeriod=30 Nov 04 05:44:49.188909 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.188863 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:49.637796 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.637763 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:49.698162 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.698131 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:49.698457 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.698433 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:49.699070 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:49.699054 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:44:50.220340 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.220303 5104 generic.go:356] "Generic (PLEG): container finished" podID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" containerID="f0c899ae555a776151616d7612d58ed7aeaa426aaa6303d509adf0ac351ef001" exitCode=0 Nov 04 05:44:50.220486 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.220350 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" event={"ID":"b25581a3-ef28-4b8e-a197-1d64ffc5c32f","Type":"ContainerDied","Data":"f0c899ae555a776151616d7612d58ed7aeaa426aaa6303d509adf0ac351ef001"} Nov 04 05:44:50.302905 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.302883 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:44:50.352906 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.352830 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:44:50.353076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.353011 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" podUID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" containerName="maas-api" containerID="cri-o://15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f" gracePeriod=30 Nov 04 05:44:50.367272 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.367245 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") pod \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " Nov 04 05:44:50.367427 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.367326 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\" (UniqueName: \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\") pod \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " Nov 04 05:44:50.367427 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.367361 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\") pod \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " Nov 04 05:44:50.367559 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.367472 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\") pod \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\" (UID: \"b25581a3-ef28-4b8e-a197-1d64ffc5c32f\") " Nov 04 05:44:50.368494 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.368461 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca" pod "b25581a3-ef28-4b8e-a197-1d64ffc5c32f" (UID: "b25581a3-ef28-4b8e-a197-1d64ffc5c32f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:50.373435 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.373352 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls" pod "b25581a3-ef28-4b8e-a197-1d64ffc5c32f" (UID: "b25581a3-ef28-4b8e-a197-1d64ffc5c32f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:50.373435 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.373399 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289" pod "b25581a3-ef28-4b8e-a197-1d64ffc5c32f" (UID: "b25581a3-ef28-4b8e-a197-1d64ffc5c32f"). InnerVolumeSpecName "kube-api-access-lc289". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:50.382383 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.382353 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs" pod "b25581a3-ef28-4b8e-a197-1d64ffc5c32f" (UID: "b25581a3-ef28-4b8e-a197-1d64ffc5c32f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:50.468340 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.468308 5104 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-metrics-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:50.468340 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.468336 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-maas-api-tls\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:50.468340 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.468347 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc289\" (UniqueName: \"kubernetes.io/projected/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-kube-api-access-lc289\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:50.468645 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.468359 5104 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25581a3-ef28-4b8e-a197-1d64ffc5c32f-openshift-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:50.519530 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:50.519493 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:51.227717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.227687 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" Nov 04 05:44:51.228157 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.227727 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q" event={"ID":"b25581a3-ef28-4b8e-a197-1d64ffc5c32f","Type":"ContainerDied","Data":"edce7c5e3ca6160bb90e5d6f2c894edb5bb6e973e42f5a43bd000eee7d16e089"} Nov 04 05:44:51.228157 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.227788 5104 scope.go:122] "RemoveContainer" containerID="f0c899ae555a776151616d7612d58ed7aeaa426aaa6303d509adf0ac351ef001" Nov 04 05:44:51.412601 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.412574 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:44:51.469115 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.469086 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-098178-64854f4957-ttb4q"] Nov 04 05:44:51.628285 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.628255 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:44:51.763621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.763584 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:51.793745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.793665 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\" (UniqueName: \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\") pod \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " Nov 04 05:44:51.793745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.793706 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\") pod \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " Nov 04 05:44:51.793985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.793851 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\") pod \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " Nov 04 05:44:51.793985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.793892 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") pod \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\" (UID: \"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea\") " Nov 04 05:44:51.794326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.794300 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca" pod "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" (UID: "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:51.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.796176 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs" pod "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" (UID: "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:51.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.796410 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57" pod "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" (UID: "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea"). InnerVolumeSpecName "kube-api-access-rgw57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:51.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.796448 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls" pod "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" (UID: "f2bc7488-ea75-4fa0-aec6-e0e5f1367dea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:51.895747 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.895703 5104 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-openshift-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:51.895747 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.895743 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-maas-api-tls\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:51.895944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.895758 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgw57\" (UniqueName: \"kubernetes.io/projected/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-kube-api-access-rgw57\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:51.895944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:51.895774 5104 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea-metrics-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:52.167150 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.162772 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:44:52.167150 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.163016 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" containerID="cri-o://0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c" gracePeriod=30 Nov 04 05:44:52.235923 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.235888 5104 generic.go:356] "Generic (PLEG): container finished" podID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" containerID="15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f" exitCode=0 Nov 04 05:44:52.236341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.235931 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" Nov 04 05:44:52.236341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.235973 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" event={"ID":"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea","Type":"ContainerDied","Data":"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f"} Nov 04 05:44:52.236341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.236026 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k" event={"ID":"f2bc7488-ea75-4fa0-aec6-e0e5f1367dea","Type":"ContainerDied","Data":"ec4138b8099e79c6a7135e6ed0b337a6d59f68ea8ab56466a4f5f0244dfccc87"} Nov 04 05:44:52.236341 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.236054 5104 scope.go:122] "RemoveContainer" containerID="15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f" Nov 04 05:44:52.245796 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.245774 5104 scope.go:122] "RemoveContainer" containerID="15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f" Nov 04 05:44:52.246056 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:52.246040 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f\": container with ID starting with 15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f not found: ID does not exist" containerID="15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f" Nov 04 05:44:52.246106 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.246063 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f"} err="failed to get container status \"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f\": rpc error: code = NotFound desc = could not find container \"15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f\": container with ID starting with 15b853d2af211d72802940ecd83575fad3c3604b2ad327b1c1905acdb3d3558f not found: ID does not exist" Nov 04 05:44:52.424440 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.424364 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:44:52.431303 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.431274 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-283815-78ff4d968f-g2c9k"] Nov 04 05:44:52.552270 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.552237 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" path="/var/lib/kubelet/pods/b25581a3-ef28-4b8e-a197-1d64ffc5c32f/volumes" Nov 04 05:44:52.552703 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:52.552682 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" path="/var/lib/kubelet/pods/f2bc7488-ea75-4fa0-aec6-e0e5f1367dea/volumes" Nov 04 05:44:53.768241 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:53.768201 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:54.072416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:54.072322 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:54.831677 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:54.831644 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:55.043445 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:55.043396 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:55.262248 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:55.261486 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:57.039236 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.039194 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:57.256620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.256581 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:57.452624 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.452597 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:44:57.482305 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482271 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482489 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482325 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482489 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482382 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482489 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482419 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482660 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482510 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482660 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482650 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482765 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482637 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:57.482765 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482681 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:57.482765 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482690 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482765 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482728 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482765 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\") pod \"5f091441-e5dc-4e75-9939-9ff758997808\" (UID: \"5f091441-e5dc-4e75-9939-9ff758997808\") " Nov 04 05:44:57.482968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482820 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:57.482968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.482885 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:57.483166 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.483146 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.483243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.483173 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.483243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.483187 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.483243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.483201 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.483419 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.483355 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:57.485079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.485046 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:44:57.485203 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.485091 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:57.485203 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.485114 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:57.485384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.485337 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm" pod "5f091441-e5dc-4e75-9939-9ff758997808" (UID: "5f091441-e5dc-4e75-9939-9ff758997808"). InnerVolumeSpecName "kube-api-access-tr2hm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:57.584013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.583888 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f091441-e5dc-4e75-9939-9ff758997808-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.584013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.583919 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.584013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.583933 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr2hm\" (UniqueName: \"kubernetes.io/projected/5f091441-e5dc-4e75-9939-9ff758997808-kube-api-access-tr2hm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.584013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.583948 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f091441-e5dc-4e75-9939-9ff758997808-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.584013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.583962 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f091441-e5dc-4e75-9939-9ff758997808-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.716010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.715971 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:57.924208 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:57.924093 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:58.283923 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.283886 5104 generic.go:356] "Generic (PLEG): container finished" podID="5f091441-e5dc-4e75-9939-9ff758997808" containerID="0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c" exitCode=0 Nov 04 05:44:58.284431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.283930 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" Nov 04 05:44:58.284431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.283978 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" event={"ID":"5f091441-e5dc-4e75-9939-9ff758997808","Type":"ContainerDied","Data":"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c"} Nov 04 05:44:58.284431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.284030 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz" event={"ID":"5f091441-e5dc-4e75-9939-9ff758997808","Type":"ContainerDied","Data":"0a34c21e15fbbf297629a55ea167c5859971aaaa3726e0e6226f48f32993dc40"} Nov 04 05:44:58.284431 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.284053 5104 scope.go:122] "RemoveContainer" containerID="0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c" Nov 04 05:44:58.298507 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.298458 5104 scope.go:122] "RemoveContainer" containerID="0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c" Nov 04 05:44:58.298872 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:44:58.298839 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c\": container with ID starting with 0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c not found: ID does not exist" containerID="0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c" Nov 04 05:44:58.298980 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.298881 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c"} err="failed to get container status \"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c\": rpc error: code = NotFound desc = could not find container \"0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c\": container with ID starting with 0546b89f247b460d3ff7037bf9b8beee3dc3f88ae0b7583e240271ea5e339c2c not found: ID does not exist" Nov 04 05:44:58.400020 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.399982 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:44:58.400207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.400182 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" podUID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" containerName="maas-api" containerID="cri-o://61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2" gracePeriod=30 Nov 04 05:44:58.551081 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.551011 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:44:58.566782 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.566752 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5gwkz"] Nov 04 05:44:58.611304 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.608585 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:58.938525 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:58.938413 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:59.396097 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.396060 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:44:59.684742 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.684430 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:44:59.713161 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.713132 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\" (UniqueName: \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\") pod \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " Nov 04 05:44:59.713365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.713187 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") pod \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " Nov 04 05:44:59.713365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.713215 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\") pod \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " Nov 04 05:44:59.713365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.713360 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\") pod \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\" (UID: \"dc9baa3e-5a92-4aaa-a056-12381cc92b9f\") " Nov 04 05:44:59.713798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.713769 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca" pod "dc9baa3e-5a92-4aaa-a056-12381cc92b9f" (UID: "dc9baa3e-5a92-4aaa-a056-12381cc92b9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:59.715398 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.715365 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8" pod "dc9baa3e-5a92-4aaa-a056-12381cc92b9f" (UID: "dc9baa3e-5a92-4aaa-a056-12381cc92b9f"). InnerVolumeSpecName "kube-api-access-6dvc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:59.715687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.715663 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls" pod "dc9baa3e-5a92-4aaa-a056-12381cc92b9f" (UID: "dc9baa3e-5a92-4aaa-a056-12381cc92b9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:59.715776 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.715751 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs" pod "dc9baa3e-5a92-4aaa-a056-12381cc92b9f" (UID: "dc9baa3e-5a92-4aaa-a056-12381cc92b9f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:59.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.814237 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dvc8\" (UniqueName: \"kubernetes.io/projected/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-kube-api-access-6dvc8\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:59.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.814276 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-maas-api-tls\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:59.814284 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.814291 5104 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-metrics-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:59.814511 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:44:59.814305 5104 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9baa3e-5a92-4aaa-a056-12381cc92b9f-openshift-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:00.096439 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.096405 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:00.176190 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176150 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:00.176737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176710 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" Nov 04 05:45:00.176737 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176736 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176753 5104 cpu_manager.go:414] "containerMap: removing container" podUID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176761 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176787 5104 cpu_manager.go:414] "containerMap: removing container" podUID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176796 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176815 5104 cpu_manager.go:414] "containerMap: removing container" podUID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176822 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176920 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2bc7488-ea75-4fa0-aec6-e0e5f1367dea" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176934 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" containerName="maas-api" Nov 04 05:45:00.176942 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176947 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f091441-e5dc-4e75-9939-9ff758997808" containerName="istio-proxy" Nov 04 05:45:00.177441 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.176956 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="b25581a3-ef28-4b8e-a197-1d64ffc5c32f" containerName="maas-api" Nov 04 05:45:00.180919 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.180896 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.182272 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.182248 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:00.186327 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.186305 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.218472 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.218445 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.218594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.218508 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjfl8\" (UniqueName: \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.218651 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.218606 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65vqq\" (UniqueName: \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.218760 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.218737 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.243459 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.243434 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:00.255337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.254845 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:00.316490 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.316354 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:45:00.319521 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.319491 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.319701 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.319557 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfl8\" (UniqueName: \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.319701 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.319585 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65vqq\" (UniqueName: \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.319903 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.319739 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.320374 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.320344 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.320538 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.320515 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.321107 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.321081 5104 generic.go:356] "Generic (PLEG): container finished" podID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" containerID="61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2" exitCode=0 Nov 04 05:45:00.321196 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.321177 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" event={"ID":"dc9baa3e-5a92-4aaa-a056-12381cc92b9f","Type":"ContainerDied","Data":"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2"} Nov 04 05:45:00.321276 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.321261 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" event={"ID":"dc9baa3e-5a92-4aaa-a056-12381cc92b9f","Type":"ContainerDied","Data":"59cb16e45ae0bed3698a3190b6710ba941a3aee400f7aeeb5d1c9096b170c564"} Nov 04 05:45:00.321315 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.321299 5104 scope.go:122] "RemoveContainer" containerID="61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2" Nov 04 05:45:00.321420 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.321407 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6" Nov 04 05:45:00.336956 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.336934 5104 scope.go:122] "RemoveContainer" containerID="61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2" Nov 04 05:45:00.337602 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:00.337393 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2\": container with ID starting with 61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2 not found: ID does not exist" containerID="61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2" Nov 04 05:45:00.337714 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.337634 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2"} err="failed to get container status \"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2\": rpc error: code = NotFound desc = could not find container \"61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2\": container with ID starting with 61c68ad5db9da04cc31bafcd88bbde99720a63a90ffad3dc52ee881ded0cb4a2 not found: ID does not exist" Nov 04 05:45:00.386806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.386726 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65vqq\" (UniqueName: \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\") pod \"maas-api-key-cleanup-29896185-dr65m\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.386948 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.386812 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjfl8\" (UniqueName: \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\") pod \"maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.494822 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:00.494786 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:45:00.500894 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.500860 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:00.507509 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.507474 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:00.552107 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.552075 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f091441-e5dc-4e75-9939-9ff758997808" path="/var/lib/kubelet/pods/5f091441-e5dc-4e75-9939-9ff758997808/volumes" Nov 04 05:45:00.562403 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.562367 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:45:00.579578 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.579543 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fbbad9-5dd4ddf495-jn4s6"] Nov 04 05:45:00.646751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.646620 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:00.865047 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.864899 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:00.922697 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.922669 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:00.924608 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:45:00.924573 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35d787d_88f2_4e1c_871f_6e2c138aef36.slice/crio-b9f31eae3f11baa3c56b7fa6b9d4df721d7cc3c8e04b9b0bf6436e73e9598b5d WatchSource:0}: Error finding container b9f31eae3f11baa3c56b7fa6b9d4df721d7cc3c8e04b9b0bf6436e73e9598b5d: Status 404 returned error can't find the container with id b9f31eae3f11baa3c56b7fa6b9d4df721d7cc3c8e04b9b0bf6436e73e9598b5d Nov 04 05:45:00.980916 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:00.980886 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:01.035787 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:45:01.035754 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730e245d_87b5_4165_92e9_a4c8cc578234.slice/crio-12e2dc4d716e642fd42c7e85e1f1ec89e0d8758f6e6683613aef6fea1d7ae9b1 WatchSource:0}: Error finding container 12e2dc4d716e642fd42c7e85e1f1ec89e0d8758f6e6683613aef6fea1d7ae9b1: Status 404 returned error can't find the container with id 12e2dc4d716e642fd42c7e85e1f1ec89e0d8758f6e6683613aef6fea1d7ae9b1 Nov 04 05:45:01.082193 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.082159 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:01.301124 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.301080 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:01.328596 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.328562 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerStarted","Data":"9daea4fdbc12c656259120c7b3905099cf7851b67492aa85887a4919d46034ef"} Nov 04 05:45:01.328777 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.328626 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerStarted","Data":"b9f31eae3f11baa3c56b7fa6b9d4df721d7cc3c8e04b9b0bf6436e73e9598b5d"} Nov 04 05:45:01.329993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.329962 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" event={"ID":"730e245d-87b5-4165-92e9-a4c8cc578234","Type":"ContainerStarted","Data":"12e2dc4d716e642fd42c7e85e1f1ec89e0d8758f6e6683613aef6fea1d7ae9b1"} Nov 04 05:45:01.460007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.459940 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" podStartSLOduration=1.459925919 podStartE2EDuration="1.459925919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:01.457026832 +0000 UTC m=+1039.021410131" watchObservedRunningTime="2026-11-04 05:45:01.459925919 +0000 UTC m=+1039.024309218" Nov 04 05:45:01.776719 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:01.776666 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:02.338994 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.338963 5104 generic.go:356] "Generic (PLEG): container finished" podID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerID="9daea4fdbc12c656259120c7b3905099cf7851b67492aa85887a4919d46034ef" exitCode=60 Nov 04 05:45:02.339195 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.339050 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerDied","Data":"9daea4fdbc12c656259120c7b3905099cf7851b67492aa85887a4919d46034ef"} Nov 04 05:45:02.339323 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.339305 5104 scope.go:122] "RemoveContainer" containerID="9daea4fdbc12c656259120c7b3905099cf7851b67492aa85887a4919d46034ef" Nov 04 05:45:02.340998 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.340971 5104 generic.go:356] "Generic (PLEG): container finished" podID="730e245d-87b5-4165-92e9-a4c8cc578234" containerID="a6b97b12f3b7b70fd9322c0ee66250175347d6f0c83e0a789371a36ba408cb11" exitCode=60 Nov 04 05:45:02.341073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.341003 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" event={"ID":"730e245d-87b5-4165-92e9-a4c8cc578234","Type":"ContainerDied","Data":"a6b97b12f3b7b70fd9322c0ee66250175347d6f0c83e0a789371a36ba408cb11"} Nov 04 05:45:02.341331 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.341309 5104 scope.go:122] "RemoveContainer" containerID="a6b97b12f3b7b70fd9322c0ee66250175347d6f0c83e0a789371a36ba408cb11" Nov 04 05:45:02.542508 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.542471 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:02.549004 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.548975 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc9baa3e-5a92-4aaa-a056-12381cc92b9f" path="/var/lib/kubelet/pods/dc9baa3e-5a92-4aaa-a056-12381cc92b9f/volumes" Nov 04 05:45:02.758306 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:02.758271 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:03.024945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.024912 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:03.237530 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.237493 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:03.348590 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.348559 5104 generic.go:356] "Generic (PLEG): container finished" podID="730e245d-87b5-4165-92e9-a4c8cc578234" containerID="3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5" exitCode=60 Nov 04 05:45:03.348696 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.348638 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" event={"ID":"730e245d-87b5-4165-92e9-a4c8cc578234","Type":"ContainerDied","Data":"3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5"} Nov 04 05:45:03.348735 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.348705 5104 scope.go:122] "RemoveContainer" containerID="a6b97b12f3b7b70fd9322c0ee66250175347d6f0c83e0a789371a36ba408cb11" Nov 04 05:45:03.348964 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.348950 5104 scope.go:122] "RemoveContainer" containerID="3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5" Nov 04 05:45:03.349278 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:03.349256 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj_odh-ai-gateway-infra(730e245d-87b5-4165-92e9-a4c8cc578234)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" Nov 04 05:45:03.351106 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.351085 5104 generic.go:356] "Generic (PLEG): container finished" podID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerID="1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734" exitCode=60 Nov 04 05:45:03.351186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.351126 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerDied","Data":"1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734"} Nov 04 05:45:03.351361 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.351343 5104 scope.go:122] "RemoveContainer" containerID="1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734" Nov 04 05:45:03.351531 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:03.351514 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896185-dr65m_odh-ai-gateway-infra(f35d787d-88f2-4e1c-871f-6e2c138aef36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" Nov 04 05:45:03.391674 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.391649 5104 scope.go:122] "RemoveContainer" containerID="9daea4fdbc12c656259120c7b3905099cf7851b67492aa85887a4919d46034ef" Nov 04 05:45:03.450088 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.450056 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:03.706158 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:03.706067 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:04.246906 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:04.246869 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:04.358013 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:04.357976 5104 scope.go:122] "RemoveContainer" containerID="3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5" Nov 04 05:45:04.358295 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:04.358270 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj_odh-ai-gateway-infra(730e245d-87b5-4165-92e9-a4c8cc578234)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" Nov 04 05:45:04.359635 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:04.359614 5104 scope.go:122] "RemoveContainer" containerID="1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734" Nov 04 05:45:04.359844 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:04.359824 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896185-dr65m_odh-ai-gateway-infra(f35d787d-88f2-4e1c-871f-6e2c138aef36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" Nov 04 05:45:04.700439 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:04.700348 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:05.243798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:05.243759 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:05.971902 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:05.971859 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:06.522743 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:06.522693 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:06.738592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:06.738552 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:08.693149 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:08.693114 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:12.005154 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:12.005125 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:12.212194 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:12.212153 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:14.660426 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:14.660397 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:15.542971 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:15.542942 5104 scope.go:122] "RemoveContainer" containerID="1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734" Nov 04 05:45:15.543137 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:15.543025 5104 scope.go:122] "RemoveContainer" containerID="3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5" Nov 04 05:45:16.370079 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.370052 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:16.434310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.434284 5104 generic.go:356] "Generic (PLEG): container finished" podID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerID="8f0fd19cbefcf2bc227e7936780304386173887d3eceb613c53fc7c0b71d0e08" exitCode=60 Nov 04 05:45:16.434451 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.434365 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerDied","Data":"8f0fd19cbefcf2bc227e7936780304386173887d3eceb613c53fc7c0b71d0e08"} Nov 04 05:45:16.434451 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.434421 5104 scope.go:122] "RemoveContainer" containerID="1928be0bc3d564b9256fe5da02742bc0e01d1dd57c6328ba5681362d900f2734" Nov 04 05:45:16.434809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.434788 5104 scope.go:122] "RemoveContainer" containerID="8f0fd19cbefcf2bc227e7936780304386173887d3eceb613c53fc7c0b71d0e08" Nov 04 05:45:16.435101 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:16.435079 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29896185-dr65m_odh-ai-gateway-infra(f35d787d-88f2-4e1c-871f-6e2c138aef36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" Nov 04 05:45:16.436442 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.436421 5104 generic.go:356] "Generic (PLEG): container finished" podID="730e245d-87b5-4165-92e9-a4c8cc578234" containerID="c34b73dc68d7183c708fab43fb5429c2b0f233fd433616601516dea0514d64d8" exitCode=60 Nov 04 05:45:16.436521 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.436454 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" event={"ID":"730e245d-87b5-4165-92e9-a4c8cc578234","Type":"ContainerDied","Data":"c34b73dc68d7183c708fab43fb5429c2b0f233fd433616601516dea0514d64d8"} Nov 04 05:45:16.436724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.436711 5104 scope.go:122] "RemoveContainer" containerID="c34b73dc68d7183c708fab43fb5429c2b0f233fd433616601516dea0514d64d8" Nov 04 05:45:16.436935 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:16.436919 5104 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj_odh-ai-gateway-infra(730e245d-87b5-4165-92e9-a4c8cc578234)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" Nov 04 05:45:16.451552 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.451532 5104 scope.go:122] "RemoveContainer" containerID="3f6f118ca42d142b7e17a33253b4cf24eab15f1bba361e07da88fc1ce90f77f5" Nov 04 05:45:16.570990 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:16.570956 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:17.469484 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.469451 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:17.570807 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.570775 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:17.697937 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.697513 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:17.713421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.713395 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:17.808496 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.808465 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\") pod \"f35d787d-88f2-4e1c-871f-6e2c138aef36\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " Nov 04 05:45:17.808652 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.808503 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\" (UniqueName: \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\") pod \"f35d787d-88f2-4e1c-871f-6e2c138aef36\" (UID: \"f35d787d-88f2-4e1c-871f-6e2c138aef36\") " Nov 04 05:45:17.808879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.808857 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca" pod "f35d787d-88f2-4e1c-871f-6e2c138aef36" (UID: "f35d787d-88f2-4e1c-871f-6e2c138aef36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:17.808941 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.808928 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d787d-88f2-4e1c-871f-6e2c138aef36-maas-api-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:17.811207 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.811183 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq" pod "f35d787d-88f2-4e1c-871f-6e2c138aef36" (UID: "f35d787d-88f2-4e1c-871f-6e2c138aef36"). InnerVolumeSpecName "kube-api-access-65vqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:17.846477 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.846453 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:17.910475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:17.910444 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65vqq\" (UniqueName: \"kubernetes.io/projected/f35d787d-88f2-4e1c-871f-6e2c138aef36-kube-api-access-65vqq\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:18.010772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.010695 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\" (UniqueName: \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\") pod \"730e245d-87b5-4165-92e9-a4c8cc578234\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " Nov 04 05:45:18.010930 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.010808 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\") pod \"730e245d-87b5-4165-92e9-a4c8cc578234\" (UID: \"730e245d-87b5-4165-92e9-a4c8cc578234\") " Nov 04 05:45:18.011141 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.011119 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca" pod "730e245d-87b5-4165-92e9-a4c8cc578234" (UID: "730e245d-87b5-4165-92e9-a4c8cc578234"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:18.012732 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.012712 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8" pod "730e245d-87b5-4165-92e9-a4c8cc578234" (UID: "730e245d-87b5-4165-92e9-a4c8cc578234"). InnerVolumeSpecName "kube-api-access-jjfl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:18.112005 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.111968 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/730e245d-87b5-4165-92e9-a4c8cc578234-maas-api-service-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:18.112005 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.112009 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjfl8\" (UniqueName: \"kubernetes.io/projected/730e245d-87b5-4165-92e9-a4c8cc578234-kube-api-access-jjfl8\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:18.452304 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.452209 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" Nov 04 05:45:18.452304 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.452249 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m" event={"ID":"f35d787d-88f2-4e1c-871f-6e2c138aef36","Type":"ContainerDied","Data":"b9f31eae3f11baa3c56b7fa6b9d4df721d7cc3c8e04b9b0bf6436e73e9598b5d"} Nov 04 05:45:18.452518 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.452321 5104 scope.go:122] "RemoveContainer" containerID="8f0fd19cbefcf2bc227e7936780304386173887d3eceb613c53fc7c0b71d0e08" Nov 04 05:45:18.453757 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.453738 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" event={"ID":"730e245d-87b5-4165-92e9-a4c8cc578234","Type":"ContainerDied","Data":"12e2dc4d716e642fd42c7e85e1f1ec89e0d8758f6e6683613aef6fea1d7ae9b1"} Nov 04 05:45:18.453861 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.453794 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj" Nov 04 05:45:18.462118 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.462093 5104 scope.go:122] "RemoveContainer" containerID="c34b73dc68d7183c708fab43fb5429c2b0f233fd433616601516dea0514d64d8" Nov 04 05:45:18.633291 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.633256 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:18.637474 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.637452 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-961485-29896185-qjzxj"] Nov 04 05:45:18.758004 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.757980 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:18.761969 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:18.761949 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896185-dr65m"] Nov 04 05:45:20.549983 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:20.549941 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" path="/var/lib/kubelet/pods/730e245d-87b5-4165-92e9-a4c8cc578234/volumes" Nov 04 05:45:20.550645 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:20.550624 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" path="/var/lib/kubelet/pods/f35d787d-88f2-4e1c-871f-6e2c138aef36/volumes" Nov 04 05:45:22.763871 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:22.763827 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:25.962364 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:25.962315 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:28.017243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:28.006905 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:29.111944 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:29.111896 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:29.325534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:29.325464 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:29.598646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:29.598606 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:29.806896 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:29.806839 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:30.491251 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:30.491200 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:45:30.611644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:30.611609 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:30.830292 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:30.830180 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:45:30.833575 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:30.833545 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:31.044593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:31.044541 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:32.221352 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:32.221305 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:33.266199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:33.266162 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:38.477687 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:38.477653 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:38.873250 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:38.873153 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:39.432214 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.432182 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:39.641034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.640997 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:39.783503 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783472 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:45:39.783883 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783869 5104 cpu_manager.go:414] "containerMap: removing container" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783884 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783901 5104 cpu_manager.go:414] "containerMap: removing container" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783906 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783915 5104 cpu_manager.go:414] "containerMap: removing container" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783920 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783929 5104 cpu_manager.go:414] "containerMap: removing container" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.783934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783934 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783941 5104 cpu_manager.go:414] "containerMap: removing container" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783945 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783952 5104 cpu_manager.go:414] "containerMap: removing container" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.783956 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.784047 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.784055 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.784060 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.784092 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.784319 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.784105 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="730e245d-87b5-4165-92e9-a4c8cc578234" containerName="cleanup" Nov 04 05:45:39.787555 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.787539 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825046 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825021 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825057 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825075 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825099 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825271 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825367 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825348 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825430 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825415 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825463 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.825510 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.825499 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbc6\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.845819 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.845793 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-a84d48-openshift-default-dockercfg-cqcsm\"" Nov 04 05:45:39.906634 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.906609 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:45:39.926804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926776 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.926935 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926829 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.926935 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926861 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbc6\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.926935 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926913 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927106 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926936 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927106 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.926955 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927100 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927144 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927179 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927382 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927285 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927421 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927377 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927542 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927520 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927791 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927773 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.927852 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.927837 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.929743 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.929717 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.929923 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.929901 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.993476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.993444 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:39.993840 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:39.993817 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbc6\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:40.101021 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.100929 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:40.552718 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.552667 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:45:40.552924 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.552899 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:45:40.553033 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.553010 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:45:40.556934 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.556910 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:45:40.592720 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:40.592696 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" event={"ID":"b93061fc-0f44-4426-866e-4a09ed15816a","Type":"ContainerStarted","Data":"dfb815067a296528873917114b785fe7058d286a711a8e89140b163e87338797"} Nov 04 05:45:41.600544 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:41.600506 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" event={"ID":"b93061fc-0f44-4426-866e-4a09ed15816a","Type":"ContainerStarted","Data":"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc"} Nov 04 05:45:41.789101 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:41.789037 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podStartSLOduration=2.7890238 podStartE2EDuration="2.7890238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:41.786848512 +0000 UTC m=+1079.351231835" watchObservedRunningTime="2026-11-04 05:45:41.7890238 +0000 UTC m=+1079.353407099" Nov 04 05:45:42.101809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:42.101771 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:42.103355 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:42.103332 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:42.103459 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:42.103398 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:43.101313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:43.101207 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:43.101313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:43.101295 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:44.102133 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:44.102085 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:44.102550 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:44.102138 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:44.212214 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:44.212183 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:44.878952 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:44.878911 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:45.079950 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:45.079901 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:45.101753 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:45.101725 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:45.101880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:45.101775 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:45.861620 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:45.861559 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" podUID="663de138-eb46-4baa-9885-eb8f3bf91544" containerName="main" containerID="cri-o://6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca" gracePeriod=45 Nov 04 05:45:46.102724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.102685 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:46.102889 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.102753 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:46.198999 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.198974 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:45:46.318017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.317878 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.317933 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.318021 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318328 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.318068 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\" (UniqueName: \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318328 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.318126 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\" (UniqueName: \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318328 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.318257 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\") pod \"663de138-eb46-4baa-9885-eb8f3bf91544\" (UID: \"663de138-eb46-4baa-9885-eb8f3bf91544\") " Nov 04 05:45:46.318945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.318901 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.319543 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.319519 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.319776 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.319756 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.322491 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.322469 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "kube-api-access-pg9lm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:46.325032 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.324999 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:46.327759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.327728 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm" pod "663de138-eb46-4baa-9885-eb8f3bf91544" (UID: "663de138-eb46-4baa-9885-eb8f3bf91544"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.419512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419480 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663de138-eb46-4baa-9885-eb8f3bf91544-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.419512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419504 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.419512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419513 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.419512 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419521 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.419770 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419530 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663de138-eb46-4baa-9885-eb8f3bf91544-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.419770 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.419540 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg9lm\" (UniqueName: \"kubernetes.io/projected/663de138-eb46-4baa-9885-eb8f3bf91544-kube-api-access-pg9lm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.627863 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.627824 5104 generic.go:356] "Generic (PLEG): container finished" podID="663de138-eb46-4baa-9885-eb8f3bf91544" containerID="6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca" exitCode=0 Nov 04 05:45:46.628036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.627879 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" event={"ID":"663de138-eb46-4baa-9885-eb8f3bf91544","Type":"ContainerDied","Data":"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca"} Nov 04 05:45:46.628036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.627882 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" Nov 04 05:45:46.628036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.627919 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9" event={"ID":"663de138-eb46-4baa-9885-eb8f3bf91544","Type":"ContainerDied","Data":"c91d5c8f070e12cb762aa7aeb3ffe398703eef03e824637f12b265c0e2f58298"} Nov 04 05:45:46.628036 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.627942 5104 scope.go:122] "RemoveContainer" containerID="6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca" Nov 04 05:45:46.639958 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.639931 5104 scope.go:122] "RemoveContainer" containerID="6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca" Nov 04 05:45:46.640285 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:45:46.640257 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca\": container with ID starting with 6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca not found: ID does not exist" containerID="6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca" Nov 04 05:45:46.640381 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.640297 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca"} err="failed to get container status \"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca\": rpc error: code = NotFound desc = could not find container \"6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca\": container with ID starting with 6ce6b8c7b0802bec8b5d7da489b06e6106b48658656decd4120f632d0dd1ffca not found: ID does not exist" Nov 04 05:45:46.816213 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.816101 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:45:46.819426 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:46.819394 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/premium-w3-a84d48-kserve-64dfb7bcd4-98pb9"] Nov 04 05:45:47.101706 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:47.101608 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:47.101706 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:47.101673 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Nov 04 05:45:48.549781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:48.549736 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="663de138-eb46-4baa-9885-eb8f3bf91544" path="/var/lib/kubelet/pods/663de138-eb46-4baa-9885-eb8f3bf91544/volumes" Nov 04 05:45:49.143416 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.143376 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:45:49.143662 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.143441 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:45:49.566878 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.566833 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:49.603643 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.603614 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:49.603940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.603910 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:49.604804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:49.604781 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:45:51.006102 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:51.005956 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:54.863627 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:54.863580 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:45:58.062621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:45:58.062564 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:00.257633 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:00.257586 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:00.490503 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:00.490465 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:46:04.804473 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:04.804436 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:06.768252 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:06.768206 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:07.055517 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:07.055415 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:07.411789 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:07.411680 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:07.633261 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:07.632639 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:09.413422 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:09.413389 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:12.068433 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:12.068397 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:15.018237 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:15.018200 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:15.273998 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:15.273912 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:15.605904 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:15.605818 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:16.690367 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:16.690331 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:17.173133 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:17.173098 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:46:17.173434 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:17.173367 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" containerID="cri-o://3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc" gracePeriod=30 Nov 04 05:46:17.213208 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:17.213162 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:17.668968 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:17.668882 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:19.606574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:19.606540 5104 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:46:19.606987 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:19.606607 5104 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:22.430473 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.430443 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:46:22.502906 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.502877 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.502918 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.502952 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.502988 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503019 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503066 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503054 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503359 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503083 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503359 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503126 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503359 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503175 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.503359 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503193 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\") pod \"b93061fc-0f44-4426-866e-4a09ed15816a\" (UID: \"b93061fc-0f44-4426-866e-4a09ed15816a\") " Nov 04 05:46:22.503572 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503350 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.503630 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503531 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:22.503693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503632 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.503693 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503653 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.503808 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503705 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.503808 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.503738 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.505282 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.505249 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.505549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.505526 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.505636 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.505538 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:22.505636 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.505601 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6" pod "b93061fc-0f44-4426-866e-4a09ed15816a" (UID: "b93061fc-0f44-4426-866e-4a09ed15816a"). InnerVolumeSpecName "kube-api-access-txbc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.604316 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604287 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txbc6\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-kube-api-access-txbc6\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604316 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604317 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604331 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604345 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b93061fc-0f44-4426-866e-4a09ed15816a-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604356 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b93061fc-0f44-4426-866e-4a09ed15816a-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604370 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b93061fc-0f44-4426-866e-4a09ed15816a-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.604481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.604382 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b93061fc-0f44-4426-866e-4a09ed15816a-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.849659 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.849571 5104 generic.go:356] "Generic (PLEG): container finished" podID="b93061fc-0f44-4426-866e-4a09ed15816a" containerID="3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc" exitCode=0 Nov 04 05:46:22.849659 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.849633 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" Nov 04 05:46:22.849851 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.849653 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" event={"ID":"b93061fc-0f44-4426-866e-4a09ed15816a","Type":"ContainerDied","Data":"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc"} Nov 04 05:46:22.849851 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.849707 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq" event={"ID":"b93061fc-0f44-4426-866e-4a09ed15816a","Type":"ContainerDied","Data":"dfb815067a296528873917114b785fe7058d286a711a8e89140b163e87338797"} Nov 04 05:46:22.849851 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.849729 5104 scope.go:122] "RemoveContainer" containerID="3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc" Nov 04 05:46:22.858583 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.858566 5104 scope.go:122] "RemoveContainer" containerID="3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc" Nov 04 05:46:22.858848 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:22.858826 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc\": container with ID starting with 3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc not found: ID does not exist" containerID="3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc" Nov 04 05:46:22.858918 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:22.858859 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc"} err="failed to get container status \"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc\": rpc error: code = NotFound desc = could not find container \"3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc\": container with ID starting with 3dced69a82b228eb00d32b0948d7be9bbd6033d0d43b700df5d2dd74426e42bc not found: ID does not exist" Nov 04 05:46:23.030994 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:23.030959 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:46:23.035849 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:23.035827 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-k5mpq"] Nov 04 05:46:23.086952 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:23.086915 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:24.550409 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:24.550373 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" path="/var/lib/kubelet/pods/b93061fc-0f44-4426-866e-4a09ed15816a/volumes" Nov 04 05:46:27.610829 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:27.610782 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:28.288467 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:28.288428 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:30.180458 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:30.180421 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:30.490384 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:30.490349 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:46:32.531131 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:32.531098 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:46:32.586582 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:32.586544 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:46:32.675420 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:32.675386 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:33.654412 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:33.654375 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:33.862383 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:33.862351 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:36.270007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:36.267575 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:36.476267 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:36.476238 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:38.471104 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:38.471073 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:46:38.528932 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:38.528737 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:46:38.621470 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:38.621433 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:38.827191 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:38.827092 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:42.823477 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:42.823444 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:43.961819 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:43.961780 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:44.925727 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:44.925680 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:47.552863 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.552815 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" podUID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" containerName="main" containerID="cri-o://a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d" gracePeriod=45 Nov 04 05:46:47.605898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.605852 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerName="main" containerID="cri-o://c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371" gracePeriod=45 Nov 04 05:46:47.610424 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.610398 5104 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": EOF" Nov 04 05:46:47.866391 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.866359 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:46:47.892311 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.892284 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:46:47.989885 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.989848 5104 generic.go:356] "Generic (PLEG): container finished" podID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" containerID="a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d" exitCode=0 Nov 04 05:46:47.990076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.989950 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" Nov 04 05:46:47.990076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.989987 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" event={"ID":"1c40d787-40fe-4cd2-aa5d-b454ea9b961f","Type":"ContainerDied","Data":"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d"} Nov 04 05:46:47.990076 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.990058 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256" event={"ID":"1c40d787-40fe-4cd2-aa5d-b454ea9b961f","Type":"ContainerDied","Data":"c2b6901fa7001489e4604585edf24039163ab4d956466bac121030f043d8b9ac"} Nov 04 05:46:47.990290 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.990087 5104 scope.go:122] "RemoveContainer" containerID="a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d" Nov 04 05:46:47.991624 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.991602 5104 generic.go:356] "Generic (PLEG): container finished" podID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerID="c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371" exitCode=0 Nov 04 05:46:47.991750 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.991634 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" event={"ID":"ee5adc13-1070-4b55-aca9-8dcdac50f754","Type":"ContainerDied","Data":"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371"} Nov 04 05:46:47.991750 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.991660 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" event={"ID":"ee5adc13-1070-4b55-aca9-8dcdac50f754","Type":"ContainerDied","Data":"3b2133947558efb7e562438a2e3a3fe977e59ca3cb8807f0d790ae8f4f76ae2a"} Nov 04 05:46:47.991750 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:47.991717 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk" Nov 04 05:46:48.005830 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.005795 5104 scope.go:122] "RemoveContainer" containerID="a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d" Nov 04 05:46:48.007480 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:48.007448 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d\": container with ID starting with a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d not found: ID does not exist" containerID="a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d" Nov 04 05:46:48.008881 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.007500 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d"} err="failed to get container status \"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d\": rpc error: code = NotFound desc = could not find container \"a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d\": container with ID starting with a72f9a929cff3242ef58aeb6b10537ddc092ac3675154e6c91aeb1130395a23d not found: ID does not exist" Nov 04 05:46:48.009000 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.008904 5104 scope.go:122] "RemoveContainer" containerID="c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371" Nov 04 05:46:48.019759 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.019737 5104 scope.go:122] "RemoveContainer" containerID="c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371" Nov 04 05:46:48.020092 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:48.020068 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371\": container with ID starting with c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371 not found: ID does not exist" containerID="c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371" Nov 04 05:46:48.020186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.020097 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371"} err="failed to get container status \"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371\": rpc error: code = NotFound desc = could not find container \"c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371\": container with ID starting with c9dd2df301bf3ae50a829273276cbe15d9160a755ca2d425699d014afa03a371 not found: ID does not exist" Nov 04 05:46:48.049692 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049667 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.049842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049703 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\" (UniqueName: \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.049842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049732 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.049842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049780 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.049842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049810 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049847 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\" (UniqueName: \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049885 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049929 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049958 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.049983 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\") pod \"ee5adc13-1070-4b55-aca9-8dcdac50f754\" (UID: \"ee5adc13-1070-4b55-aca9-8dcdac50f754\") " Nov 04 05:46:48.050067 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050044 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\" (UniqueName: \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.050436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050105 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.050436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050125 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.050436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050135 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\") pod \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\" (UID: \"1c40d787-40fe-4cd2-aa5d-b454ea9b961f\") " Nov 04 05:46:48.050436 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050319 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.050646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050513 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.050646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050535 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.050646 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050548 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.050797 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050749 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.050985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050961 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.051062 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.050983 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.054710 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.053122 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:48.054710 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.053301 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "kube-api-access-k64xb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:48.054710 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.053513 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.055089 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.055059 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:48.055146 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.055068 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm" pod "1c40d787-40fe-4cd2-aa5d-b454ea9b961f" (UID: "1c40d787-40fe-4cd2-aa5d-b454ea9b961f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:48.055473 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.055452 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7" pod "ee5adc13-1070-4b55-aca9-8dcdac50f754" (UID: "ee5adc13-1070-4b55-aca9-8dcdac50f754"). InnerVolumeSpecName "kube-api-access-pxcn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151572 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151607 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k64xb\" (UniqueName: \"kubernetes.io/projected/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-kube-api-access-k64xb\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151623 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151636 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151646 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151656 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c40d787-40fe-4cd2-aa5d-b454ea9b961f-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151664 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee5adc13-1070-4b55-aca9-8dcdac50f754-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151672 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxcn7\" (UniqueName: \"kubernetes.io/projected/ee5adc13-1070-4b55-aca9-8dcdac50f754-kube-api-access-pxcn7\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.151673 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.151680 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee5adc13-1070-4b55-aca9-8dcdac50f754-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.424137 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.424054 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:46:48.428434 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.428404 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-961485/premium-w2-961485-kserve-7d9db758dc-4h256"] Nov 04 05:46:48.552083 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.552041 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" path="/var/lib/kubelet/pods/1c40d787-40fe-4cd2-aa5d-b454ea9b961f/volumes" Nov 04 05:46:48.552468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.552447 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:46:48.552558 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:48.552491 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-961485/simulator-w2-961485-kserve-5dd8c84d6-2vpnk"] Nov 04 05:46:49.129822 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:49.129780 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:49.373900 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:49.373860 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:49.705970 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:49.705937 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:49.926358 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:49.926316 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:50.167243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.166916 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t"] Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167508 5104 cpu_manager.go:414] "containerMap: removing container" podUID="663de138-eb46-4baa-9885-eb8f3bf91544" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167528 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="663de138-eb46-4baa-9885-eb8f3bf91544" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167552 5104 cpu_manager.go:414] "containerMap: removing container" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167560 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167584 5104 cpu_manager.go:414] "containerMap: removing container" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167592 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167613 5104 cpu_manager.go:414] "containerMap: removing container" podUID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167622 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167718 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="663de138-eb46-4baa-9885-eb8f3bf91544" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167731 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="f35d787d-88f2-4e1c-871f-6e2c138aef36" containerName="cleanup" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167741 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="b93061fc-0f44-4426-866e-4a09ed15816a" containerName="istio-proxy" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167754 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" containerName="main" Nov 04 05:46:50.167798 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.167765 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c40d787-40fe-4cd2-aa5d-b454ea9b961f" containerName="main" Nov 04 05:46:50.174184 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.174159 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.231853 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.231813 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t"] Nov 04 05:46:50.279018 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.278979 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.279313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.279286 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59jsg\" (UniqueName: \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.279547 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.279528 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.287955 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.287932 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:46:50.288349 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.288330 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:46:50.395931 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.395871 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.396143 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.395948 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59jsg\" (UniqueName: \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.396143 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.396047 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.397034 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.397007 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.401859 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.401830 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.477199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.477161 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59jsg\" (UniqueName: \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\") pod \"maas-api-revoke-sub-7611e660-hqd2t\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.501142 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.500889 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:50.561365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.561330 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee5adc13-1070-4b55-aca9-8dcdac50f754" path="/var/lib/kubelet/pods/ee5adc13-1070-4b55-aca9-8dcdac50f754/volumes" Nov 04 05:46:50.950243 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:50.943292 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t"] Nov 04 05:46:50.950243 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:46:50.944270 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd829a495_9044_425f_af68_6b0f66ab99d1.slice/crio-c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b WatchSource:0}: Error finding container c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b: Status 404 returned error can't find the container with id c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b Nov 04 05:46:51.010592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:51.010560 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" event={"ID":"d829a495-9044-425f-af68-6b0f66ab99d1","Type":"ContainerStarted","Data":"c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b"} Nov 04 05:46:51.112720 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:51.112685 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:52.017358 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:52.017328 5104 generic.go:356] "Generic (PLEG): container finished" podID="d829a495-9044-425f-af68-6b0f66ab99d1" containerID="05ce2439455e4ae0ef60716a0eb3de146eda5d8b5ded3748a3a9cfd7b931609e" exitCode=0 Nov 04 05:46:52.017733 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:52.017415 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" event={"ID":"d829a495-9044-425f-af68-6b0f66ab99d1","Type":"ContainerDied","Data":"05ce2439455e4ae0ef60716a0eb3de146eda5d8b5ded3748a3a9cfd7b931609e"} Nov 04 05:46:52.383333 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:52.383299 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:52.602311 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:52.602267 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:53.163592 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.163567 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:53.215018 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.214992 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\") pod \"d829a495-9044-425f-af68-6b0f66ab99d1\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " Nov 04 05:46:53.215135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.215037 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\") pod \"d829a495-9044-425f-af68-6b0f66ab99d1\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " Nov 04 05:46:53.215135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.215068 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\" (UniqueName: \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\") pod \"d829a495-9044-425f-af68-6b0f66ab99d1\" (UID: \"d829a495-9044-425f-af68-6b0f66ab99d1\") " Nov 04 05:46:53.215366 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.215341 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca" pod "d829a495-9044-425f-af68-6b0f66ab99d1" (UID: "d829a495-9044-425f-af68-6b0f66ab99d1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:53.217123 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.217096 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg" pod "d829a495-9044-425f-af68-6b0f66ab99d1" (UID: "d829a495-9044-425f-af68-6b0f66ab99d1"). InnerVolumeSpecName "kube-api-access-59jsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:53.217212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.217137 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca" pod "d829a495-9044-425f-af68-6b0f66ab99d1" (UID: "d829a495-9044-425f-af68-6b0f66ab99d1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:53.316751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.316659 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-openshift-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.316751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.316701 5104 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d829a495-9044-425f-af68-6b0f66ab99d1-maas-api-xks-ca\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.316751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.316717 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59jsg\" (UniqueName: \"kubernetes.io/projected/d829a495-9044-425f-af68-6b0f66ab99d1-kube-api-access-59jsg\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.491163 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.491116 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" podUID="decfcd7c-7e35-4b21-b096-fad337c354ca" containerName="main" containerID="cri-o://616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0" gracePeriod=45 Nov 04 05:46:53.551993 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.551957 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" podUID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" containerName="main" containerID="cri-o://2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407" gracePeriod=45 Nov 04 05:46:53.823264 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.823216 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:46:53.839686 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.839661 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:46:53.920757 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920685 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.920757 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920720 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\" (UniqueName: \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920769 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920805 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920833 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920903 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920934 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\" (UniqueName: \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.920986 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920963 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920991 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\" (UniqueName: \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\") pod \"decfcd7c-7e35-4b21-b096-fad337c354ca\" (UID: \"decfcd7c-7e35-4b21-b096-fad337c354ca\") " Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.920994 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921031 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\" (UniqueName: \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921044 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921077 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921103 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\") pod \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\" (UID: \"88fca7cd-4489-4e29-9d24-e3d7b38db4f3\") " Nov 04 05:46:53.921293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921186 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.921675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921323 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.921675 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921507 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.921780 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.921722 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.922102 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922073 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.922212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922104 5104 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-tmp-dir\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.922212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922119 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.922212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922133 5104 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-model-cache\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.922212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922147 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.922212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.922160 5104 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-home\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.923374 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923350 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.923621 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923602 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "kube-api-access-6rw99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:53.923702 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923613 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.923837 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923814 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:53.923837 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923827 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c" pod "88fca7cd-4489-4e29-9d24-e3d7b38db4f3" (UID: "88fca7cd-4489-4e29-9d24-e3d7b38db4f3"). InnerVolumeSpecName "kube-api-access-6w77c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:53.923940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:53.923850 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs" pod "decfcd7c-7e35-4b21-b096-fad337c354ca" (UID: "decfcd7c-7e35-4b21-b096-fad337c354ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:54.023638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023603 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rw99\" (UniqueName: \"kubernetes.io/projected/decfcd7c-7e35-4b21-b096-fad337c354ca-kube-api-access-6rw99\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.023638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023640 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w77c\" (UniqueName: \"kubernetes.io/projected/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-kube-api-access-6w77c\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.023879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023659 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.023879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023671 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fca7cd-4489-4e29-9d24-e3d7b38db4f3-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.023879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023680 5104 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/decfcd7c-7e35-4b21-b096-fad337c354ca-dshm\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.023879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.023687 5104 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/decfcd7c-7e35-4b21-b096-fad337c354ca-tls-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.031751 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.031729 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" Nov 04 05:46:54.031878 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.031765 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7611e660-hqd2t" event={"ID":"d829a495-9044-425f-af68-6b0f66ab99d1","Type":"ContainerDied","Data":"c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b"} Nov 04 05:46:54.031878 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.031823 5104 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c64b3263c7767659a25f7384cf959d892e4efcf28cefe424135adba718e2796b" Nov 04 05:46:54.033164 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.033143 5104 generic.go:356] "Generic (PLEG): container finished" podID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" containerID="2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407" exitCode=0 Nov 04 05:46:54.033313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.033187 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" Nov 04 05:46:54.033313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.033254 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" event={"ID":"88fca7cd-4489-4e29-9d24-e3d7b38db4f3","Type":"ContainerDied","Data":"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407"} Nov 04 05:46:54.033313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.033295 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct" event={"ID":"88fca7cd-4489-4e29-9d24-e3d7b38db4f3","Type":"ContainerDied","Data":"4a0fcc3ab740ff2d5d8c0e3181e5bf0f22d2a08141e4002cbcc9ec34d71c4353"} Nov 04 05:46:54.033534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.033323 5104 scope.go:122] "RemoveContainer" containerID="2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407" Nov 04 05:46:54.034719 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.034700 5104 generic.go:356] "Generic (PLEG): container finished" podID="decfcd7c-7e35-4b21-b096-fad337c354ca" containerID="616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0" exitCode=0 Nov 04 05:46:54.034772 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.034751 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" event={"ID":"decfcd7c-7e35-4b21-b096-fad337c354ca","Type":"ContainerDied","Data":"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0"} Nov 04 05:46:54.034821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.034771 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" event={"ID":"decfcd7c-7e35-4b21-b096-fad337c354ca","Type":"ContainerDied","Data":"ba9f02e29d7c99f5f377b8826c9af970d86ca1261113072be0c0262a1ebba3a3"} Nov 04 05:46:54.034821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.034815 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj" Nov 04 05:46:54.042987 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.042946 5104 scope.go:122] "RemoveContainer" containerID="2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407" Nov 04 05:46:54.043334 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:54.043303 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407\": container with ID starting with 2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407 not found: ID does not exist" containerID="2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407" Nov 04 05:46:54.043453 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.043341 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407"} err="failed to get container status \"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407\": rpc error: code = NotFound desc = could not find container \"2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407\": container with ID starting with 2024ce2e4cef502fdaa7aee3798d4c79aaf90df61f4046bdf3e2ccda77d99407 not found: ID does not exist" Nov 04 05:46:54.043453 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.043363 5104 scope.go:122] "RemoveContainer" containerID="616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0" Nov 04 05:46:54.054705 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.054689 5104 scope.go:122] "RemoveContainer" containerID="616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0" Nov 04 05:46:54.054975 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:46:54.054959 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0\": container with ID starting with 616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0 not found: ID does not exist" containerID="616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0" Nov 04 05:46:54.055027 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.054980 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0"} err="failed to get container status \"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0\": rpc error: code = NotFound desc = could not find container \"616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0\": container with ID starting with 616d752420644b443e874098bd555667680f0ee468dc1d30e83efb5ca688ecc0 not found: ID does not exist" Nov 04 05:46:54.206293 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.206201 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:46:54.210869 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.210841 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/premium-w4-901a00-kserve-d68c67695-4pbct"] Nov 04 05:46:54.331405 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.331372 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:46:54.335384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.335354 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-w4-901a00-kserve-5bb5ffc674-qxrgj"] Nov 04 05:46:54.549505 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.549472 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" path="/var/lib/kubelet/pods/88fca7cd-4489-4e29-9d24-e3d7b38db4f3/volumes" Nov 04 05:46:54.549873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.549860 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="decfcd7c-7e35-4b21-b096-fad337c354ca" path="/var/lib/kubelet/pods/decfcd7c-7e35-4b21-b096-fad337c354ca/volumes" Nov 04 05:46:54.944880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:54.944793 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:55.154882 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:55.154848 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:46:57.122581 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:46:57.122545 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:00.490375 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:00.490335 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:47:02.947199 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:02.947158 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:04.976285 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:04.976216 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:06.261189 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:06.261147 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:07.226125 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:07.226090 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:08.749051 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:08.748969 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:11.134969 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:11.134925 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:11.344982 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:11.344934 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:12.294985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.294950 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295363 5104 cpu_manager.go:414] "containerMap: removing container" podUID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" containerName="main" Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295376 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" containerName="main" Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295399 5104 cpu_manager.go:414] "containerMap: removing container" podUID="d829a495-9044-425f-af68-6b0f66ab99d1" containerName="revoke-keys" Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295403 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d829a495-9044-425f-af68-6b0f66ab99d1" containerName="revoke-keys" Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295417 5104 cpu_manager.go:414] "containerMap: removing container" podUID="decfcd7c-7e35-4b21-b096-fad337c354ca" containerName="main" Nov 04 05:47:12.295432 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295423 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="decfcd7c-7e35-4b21-b096-fad337c354ca" containerName="main" Nov 04 05:47:12.295650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295482 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="88fca7cd-4489-4e29-9d24-e3d7b38db4f3" containerName="main" Nov 04 05:47:12.295650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295492 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="d829a495-9044-425f-af68-6b0f66ab99d1" containerName="revoke-keys" Nov 04 05:47:12.295650 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.295499 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="decfcd7c-7e35-4b21-b096-fad337c354ca" containerName="main" Nov 04 05:47:12.298629 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.298610 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391440 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391403 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391613 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391446 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391613 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391527 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391613 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391570 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391633 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391673 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391693 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391724 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391721 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.391871 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.391756 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbjcz\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.419161 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.419136 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:12.492748 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492715 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.492748 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492757 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492775 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492799 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492828 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492870 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492900 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493010 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.492939 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493177 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493261 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbjcz\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493276 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493360 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493268 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493576 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493419 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.493943 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.493924 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.495474 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.495440 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.495579 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.495553 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.560456 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.560384 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.560456 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.560388 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbjcz\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.611337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.611305 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:12.644557 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.644525 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:12.649180 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.649155 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.680997 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.680970 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:12.685249 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.685128 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.695833 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.695894 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.695935 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696039 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696075 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696159 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696188 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9qjc\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696286 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.698099 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.696349 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.712788 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.712670 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:12.720912 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.720887 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:12.796828 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.796790 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.796837 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.796867 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkvn8\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797009 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.796985 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797025 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797054 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797081 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797204 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797268 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797254 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797285 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797336 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797362 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797393 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797201 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797423 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797450 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797509 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797691 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797748 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797795 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.797938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797865 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.797938 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.797893 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9qjc\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.798508 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.798484 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.800318 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.800289 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.800438 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.800420 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.803636 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.803615 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:12.869876 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.869802 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.870014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.869992 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9qjc\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.899142 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899107 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899151 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkvn8\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899195 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899235 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899265 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899313 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899306 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899591 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899355 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899591 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899388 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899591 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899440 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899756 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899626 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899815 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899791 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.899935 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.899907 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.900133 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.900109 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.900334 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.900315 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.901789 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.901769 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.901891 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.901818 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.963891 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.963864 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:12.973094 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.973068 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:12.973262 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:12.973216 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkvn8\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:13.014281 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.014252 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:13.067049 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.066757 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:13.072388 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:47:13.072359 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5255c316_71d7_4bca_9daf_91c961e6116e.slice/crio-01210751d92205591baf7c6a04330b86d847095a3761b0f92db6add5e118ffc2 WatchSource:0}: Error finding container 01210751d92205591baf7c6a04330b86d847095a3761b0f92db6add5e118ffc2: Status 404 returned error can't find the container with id 01210751d92205591baf7c6a04330b86d847095a3761b0f92db6add5e118ffc2 Nov 04 05:47:13.075570 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.075531 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.075661 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.075619 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.075709 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.075660 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.196240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.192538 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" event={"ID":"5255c316-71d7-4bca-9daf-91c961e6116e","Type":"ContainerStarted","Data":"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc"} Nov 04 05:47:13.196240 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.192606 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" event={"ID":"5255c316-71d7-4bca-9daf-91c961e6116e","Type":"ContainerStarted","Data":"01210751d92205591baf7c6a04330b86d847095a3761b0f92db6add5e118ffc2"} Nov 04 05:47:13.203238 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.199397 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:13.213245 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.204695 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:13.213245 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.212610 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:47:13.213245 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.212805 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" containerID="cri-o://e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf" gracePeriod=30 Nov 04 05:47:13.300198 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.300160 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:13.327411 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.327349 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podStartSLOduration=1.327331657 podStartE2EDuration="1.327331657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:13.324190468 +0000 UTC m=+1170.888573780" watchObservedRunningTime="2026-11-04 05:47:13.327331657 +0000 UTC m=+1170.891714957" Nov 04 05:47:13.365926 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.365902 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:13.367914 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:47:13.367883 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f4221b_5705_4352_8bbe_b5cf3c2dba4c.slice/crio-d55fbbd72ef16467a59872f5d96b370c472fe108500c4e929c68346b81142117 WatchSource:0}: Error finding container d55fbbd72ef16467a59872f5d96b370c472fe108500c4e929c68346b81142117: Status 404 returned error can't find the container with id d55fbbd72ef16467a59872f5d96b370c472fe108500c4e929c68346b81142117 Nov 04 05:47:13.370105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.370076 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.370182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.370136 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.370182 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.370165 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.441557 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:47:13.435729 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f87b91a_8f51_48bc_ac01_e2c9b883cc14.slice/crio-4c48f1c77de98f13525bc346f16af1f790cac1f79738483a5e9185411b06e9b4 WatchSource:0}: Error finding container 4c48f1c77de98f13525bc346f16af1f790cac1f79738483a5e9185411b06e9b4: Status 404 returned error can't find the container with id 4c48f1c77de98f13525bc346f16af1f790cac1f79738483a5e9185411b06e9b4 Nov 04 05:47:13.441557 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.437263 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:13.448898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.443287 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.448898 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.443439 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.458347 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.458071 5104 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:47:13.512135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.512102 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:13.612088 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:13.612050 5104 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:14.201030 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.200993 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" event={"ID":"7f87b91a-8f51-48bc-ac01-e2c9b883cc14","Type":"ContainerStarted","Data":"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229"} Nov 04 05:47:14.201266 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.201051 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" event={"ID":"7f87b91a-8f51-48bc-ac01-e2c9b883cc14","Type":"ContainerStarted","Data":"4c48f1c77de98f13525bc346f16af1f790cac1f79738483a5e9185411b06e9b4"} Nov 04 05:47:14.201266 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.201060 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" podUID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" containerName="istio-proxy" containerID="cri-o://30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229" gracePeriod=30 Nov 04 05:47:14.202740 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.202715 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" event={"ID":"70f4221b-5705-4352-8bbe-b5cf3c2dba4c","Type":"ContainerStarted","Data":"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7"} Nov 04 05:47:14.202821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.202762 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" event={"ID":"70f4221b-5705-4352-8bbe-b5cf3c2dba4c","Type":"ContainerStarted","Data":"d55fbbd72ef16467a59872f5d96b370c472fe108500c4e929c68346b81142117"} Nov 04 05:47:14.202821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.202743 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" podUID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" containerName="istio-proxy" containerID="cri-o://0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7" gracePeriod=30 Nov 04 05:47:14.343244 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.341733 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" podStartSLOduration=2.341712862 podStartE2EDuration="2.341712862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:14.339541964 +0000 UTC m=+1171.903925264" watchObservedRunningTime="2026-11-04 05:47:14.341712862 +0000 UTC m=+1171.906096163" Nov 04 05:47:14.471689 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.471637 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" podStartSLOduration=2.471622532 podStartE2EDuration="2.471622532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:14.470409596 +0000 UTC m=+1172.034792896" watchObservedRunningTime="2026-11-04 05:47:14.471622532 +0000 UTC m=+1172.036005828" Nov 04 05:47:14.613325 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.613284 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:14.613473 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:14.613343 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:15.612443 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:15.612407 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:15.612945 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:15.612468 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:15.923043 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:15.922950 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:16.140253 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:16.140051 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:16.612665 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:16.612618 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:16.613158 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:16.612680 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:17.001135 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:17.001105 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:17.214071 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:17.214034 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:17.612409 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:17.612369 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:17.612574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:17.612428 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:18.513695 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.513634 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:47:18.600545 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600515 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600692 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600595 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600692 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600642 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600692 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600681 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600728 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600767 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.600873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600785 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.600873 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600859 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.601085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600866 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.601085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600899 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.601085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600960 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\") pod \"d66cbeee-0751-4aca-af9d-73e1c7c77901\" (UID: \"d66cbeee-0751-4aca-af9d-73e1c7c77901\") " Nov 04 05:47:18.601085 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.600971 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.601337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601096 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:18.601337 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601283 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.601447 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601357 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.601447 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601389 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d66cbeee-0751-4aca-af9d-73e1c7c77901-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.601447 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601403 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.601447 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.601413 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.603092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.603060 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.603092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.603079 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:18.603307 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.603286 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.603399 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.603383 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5" pod "d66cbeee-0751-4aca-af9d-73e1c7c77901" (UID: "d66cbeee-0751-4aca-af9d-73e1c7c77901"). InnerVolumeSpecName "kube-api-access-48vq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.612560 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.612536 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:18.612637 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.612590 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:18.702485 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.702449 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48vq5\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-kube-api-access-48vq5\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702485 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.702478 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702485 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.702488 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702700 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.702498 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702700 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:18.702507 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d66cbeee-0751-4aca-af9d-73e1c7c77901-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.234842 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.234808 5104 generic.go:356] "Generic (PLEG): container finished" podID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerID="e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf" exitCode=0 Nov 04 05:47:19.234940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.234872 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" Nov 04 05:47:19.234940 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.234893 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" event={"ID":"d66cbeee-0751-4aca-af9d-73e1c7c77901","Type":"ContainerDied","Data":"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf"} Nov 04 05:47:19.235027 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.234958 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2" event={"ID":"d66cbeee-0751-4aca-af9d-73e1c7c77901","Type":"ContainerDied","Data":"1da0a015bee315fb932f69c751b244a5ed921f2f7fb683c5dde7d9553fc7116c"} Nov 04 05:47:19.235027 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.234981 5104 scope.go:122] "RemoveContainer" containerID="e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf" Nov 04 05:47:19.310611 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.310578 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:19.341014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.340974 5104 scope.go:122] "RemoveContainer" containerID="e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf" Nov 04 05:47:19.341421 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:19.341398 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf\": container with ID starting with e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf not found: ID does not exist" containerID="e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf" Nov 04 05:47:19.341538 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.341430 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf"} err="failed to get container status \"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf\": rpc error: code = NotFound desc = could not find container \"e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf\": container with ID starting with e9d8b03b2d08f33d6434f7859bc08b5d51771e89d61e4d47ec72dcc20b26cabf not found: ID does not exist" Nov 04 05:47:19.459502 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.459422 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:47:19.463567 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.463350 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:19.485080 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.485058 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:19.516277 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.516245 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-rv5l2"] Nov 04 05:47:19.611877 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.611815 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 05:47:19.612045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.611892 5104 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 05:47:19.612045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.611959 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.611996 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612045 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612014 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612048 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612082 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612160 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612200 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612257 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612310 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612296 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612330 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612336 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612351 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612397 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612439 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612474 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612544 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612551 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612587 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.612619 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612619 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612657 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\") pod \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\" (UID: \"70f4221b-5705-4352-8bbe-b5cf3c2dba4c\") " Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612691 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\") pod \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\" (UID: \"7f87b91a-8f51-48bc-ac01-e2c9b883cc14\") " Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612758 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.612929 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613024 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613064 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.613082 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613081 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.613463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613097 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.613463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613112 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.613463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613127 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.613463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613178 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.613463 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613203 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:19.613777 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613461 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.613839 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.613791 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:19.614985 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.614948 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:19.615114 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.615052 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "kube-api-access-c9qjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.615541 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.615506 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.615821 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.615800 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "kube-api-access-tkvn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.616943 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.616921 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.618388 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.618360 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:19.618455 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.618381 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy" pod "70f4221b-5705-4352-8bbe-b5cf3c2dba4c" (UID: "70f4221b-5705-4352-8bbe-b5cf3c2dba4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.618455 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.618384 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy" pod "7f87b91a-8f51-48bc-ac01-e2c9b883cc14" (UID: "7f87b91a-8f51-48bc-ac01-e2c9b883cc14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713744 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713782 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713797 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713811 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9qjc\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-kube-api-access-c9qjc\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713826 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713840 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713853 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713867 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713881 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713895 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkvn8\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-kube-api-access-tkvn8\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713910 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713923 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f87b91a-8f51-48bc-ac01-e2c9b883cc14-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.713962 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.713936 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70f4221b-5705-4352-8bbe-b5cf3c2dba4c-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.741542 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.741424 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:19.742644 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.742301 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" containerID="cri-o://e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc" gracePeriod=30 Nov 04 05:47:19.765134 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.765101 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:47:19.765402 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.765376 5104 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" containerID="cri-o://989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d" gracePeriod=30 Nov 04 05:47:19.823280 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:19.820919 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:20.243783 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.243741 5104 generic.go:356] "Generic (PLEG): container finished" podID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" containerID="30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229" exitCode=0 Nov 04 05:47:20.243978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.243794 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" event={"ID":"7f87b91a-8f51-48bc-ac01-e2c9b883cc14","Type":"ContainerDied","Data":"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229"} Nov 04 05:47:20.243978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.243825 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" Nov 04 05:47:20.243978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.243852 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk" event={"ID":"7f87b91a-8f51-48bc-ac01-e2c9b883cc14","Type":"ContainerDied","Data":"4c48f1c77de98f13525bc346f16af1f790cac1f79738483a5e9185411b06e9b4"} Nov 04 05:47:20.243978 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.243882 5104 scope.go:122] "RemoveContainer" containerID="30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229" Nov 04 05:47:20.245764 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.245738 5104 generic.go:356] "Generic (PLEG): container finished" podID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" containerID="0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7" exitCode=0 Nov 04 05:47:20.245901 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.245796 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" Nov 04 05:47:20.245901 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.245840 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" event={"ID":"70f4221b-5705-4352-8bbe-b5cf3c2dba4c","Type":"ContainerDied","Data":"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7"} Nov 04 05:47:20.245901 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.245883 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx" event={"ID":"70f4221b-5705-4352-8bbe-b5cf3c2dba4c","Type":"ContainerDied","Data":"d55fbbd72ef16467a59872f5d96b370c472fe108500c4e929c68346b81142117"} Nov 04 05:47:20.255734 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.255708 5104 scope.go:122] "RemoveContainer" containerID="30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229" Nov 04 05:47:20.256135 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:20.256097 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229\": container with ID starting with 30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229 not found: ID does not exist" containerID="30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229" Nov 04 05:47:20.256326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.256133 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229"} err="failed to get container status \"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229\": rpc error: code = NotFound desc = could not find container \"30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229\": container with ID starting with 30d0e95cf08cce62192ea52b7f644f829d02771e74b257774e96d0fe9734b229 not found: ID does not exist" Nov 04 05:47:20.256326 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.256156 5104 scope.go:122] "RemoveContainer" containerID="0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7" Nov 04 05:47:20.268093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.268071 5104 scope.go:122] "RemoveContainer" containerID="0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7" Nov 04 05:47:20.268377 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:20.268358 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7\": container with ID starting with 0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7 not found: ID does not exist" containerID="0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7" Nov 04 05:47:20.268481 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.268381 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7"} err="failed to get container status \"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7\": rpc error: code = NotFound desc = could not find container \"0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7\": container with ID starting with 0b5109ec2c51c05de4874b26f2ead02902b0210a471a4711de15dd80e7802cc7 not found: ID does not exist" Nov 04 05:47:20.426603 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.426565 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:20.431987 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.431957 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-wbptx"] Nov 04 05:47:20.551253 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.551140 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" path="/var/lib/kubelet/pods/70f4221b-5705-4352-8bbe-b5cf3c2dba4c/volumes" Nov 04 05:47:20.551717 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.551591 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" path="/var/lib/kubelet/pods/d66cbeee-0751-4aca-af9d-73e1c7c77901/volumes" Nov 04 05:47:20.552153 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.552132 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:20.557972 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:20.557948 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-mzmdk"] Nov 04 05:47:21.674198 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:21.674165 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-667656475-7s2pb_f9fe744b-4bbe-4b56-a3a7-a10afc8688a9/authorino/0.log" Nov 04 05:47:22.338623 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:22.338589 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:22.548612 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:22.548582 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" path="/var/lib/kubelet/pods/7f87b91a-8f51-48bc-ac01-e2c9b883cc14/volumes" Nov 04 05:47:23.111369 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:23.111330 5104 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:47:23.111739 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:23.111388 5104 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:47:25.021105 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.021077 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:47:25.060386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060313 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060386 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060379 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060440 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060473 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060506 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060571 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060551 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060611 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060639 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060781 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060664 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\") pod \"8109cf5d-32e0-42da-9369-ba00a8207a8c\" (UID: \"8109cf5d-32e0-42da-9369-ba00a8207a8c\") " Nov 04 05:47:25.060937 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060602 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.060937 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.060824 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.061263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.061204 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.061486 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.061416 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.061549 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.061505 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.063167 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.063120 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.063167 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.063134 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.063167 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.063123 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.063353 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.063243 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk" pod "8109cf5d-32e0-42da-9369-ba00a8207a8c" (UID: "8109cf5d-32e0-42da-9369-ba00a8207a8c"). InnerVolumeSpecName "kube-api-access-pwnzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.161475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161442 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161475 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161472 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161485 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161498 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwnzk\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-kube-api-access-pwnzk\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161509 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161517 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161525 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161533 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109cf5d-32e0-42da-9369-ba00a8207a8c-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.161704 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.161545 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109cf5d-32e0-42da-9369-ba00a8207a8c-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.286865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.286448 5104 generic.go:356] "Generic (PLEG): container finished" podID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerID="989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d" exitCode=0 Nov 04 05:47:25.286865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.286575 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerDied","Data":"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d"} Nov 04 05:47:25.286865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.286660 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" event={"ID":"8109cf5d-32e0-42da-9369-ba00a8207a8c","Type":"ContainerDied","Data":"b6ebf7b33e1e01d7dc991a8d5605c2ede2519eb2d1bcdc7bee7c4f5ce3ddb117"} Nov 04 05:47:25.286865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.286703 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf" Nov 04 05:47:25.286865 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.286734 5104 scope.go:122] "RemoveContainer" containerID="989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d" Nov 04 05:47:25.296809 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.296784 5104 scope.go:122] "RemoveContainer" containerID="67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d" Nov 04 05:47:25.305520 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.305502 5104 scope.go:122] "RemoveContainer" containerID="989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d" Nov 04 05:47:25.305785 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:25.305765 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d\": container with ID starting with 989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d not found: ID does not exist" containerID="989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d" Nov 04 05:47:25.305859 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.305797 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d"} err="failed to get container status \"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d\": rpc error: code = NotFound desc = could not find container \"989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d\": container with ID starting with 989bdb27c6386f8a6ef90ac3b3f2f97945410903418c335681ef6697dd80b77d not found: ID does not exist" Nov 04 05:47:25.305859 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.305823 5104 scope.go:122] "RemoveContainer" containerID="67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d" Nov 04 05:47:25.306061 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:25.306045 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d\": container with ID starting with 67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d not found: ID does not exist" containerID="67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d" Nov 04 05:47:25.306096 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.306068 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d"} err="failed to get container status \"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d\": rpc error: code = NotFound desc = could not find container \"67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d\": container with ID starting with 67641a354ec43c8a553af4c3afee506fbe53564f967865fc137953a6605a203d not found: ID does not exist" Nov 04 05:47:25.468984 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.468951 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:47:25.473727 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.473692 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-2smlf"] Nov 04 05:47:25.893991 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.893964 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:25.968638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968606 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.968638 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968643 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.968862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968666 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.968862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968690 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.968862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968720 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.968862 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968784 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.969073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968878 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.969073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968909 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.969073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968958 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.969073 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.968967 5104 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\") pod \"5255c316-71d7-4bca-9daf-91c961e6116e\" (UID: \"5255c316-71d7-4bca-9daf-91c961e6116e\") " Nov 04 05:47:25.969365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969090 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.969365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969112 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.969365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969156 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.969365 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969175 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.969534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969366 5104 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-credential-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.969534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969387 5104 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5255c316-71d7-4bca-9daf-91c961e6116e-istiod-ca-cert\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.969534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969401 5104 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-data\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.969534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969414 5104 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-certs\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.969534 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.969427 5104 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-workload-socket\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.970925 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.970902 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.971043 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.971024 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.971444 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.971423 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.971444 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:25.971433 5104 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz" pod "5255c316-71d7-4bca-9daf-91c961e6116e" (UID: "5255c316-71d7-4bca-9daf-91c961e6116e"). InnerVolumeSpecName "kube-api-access-pbjcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:26.070211 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.070128 5104 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbjcz\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-kube-api-access-pbjcz\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.070211 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.070159 5104 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5255c316-71d7-4bca-9daf-91c961e6116e-istio-token\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.070211 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.070173 5104 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5255c316-71d7-4bca-9daf-91c961e6116e-istio-podinfo\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.070211 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.070187 5104 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5255c316-71d7-4bca-9daf-91c961e6116e-istio-envoy\") on node \"ip-10-0-139-34.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.292550 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.292510 5104 generic.go:356] "Generic (PLEG): container finished" podID="5255c316-71d7-4bca-9daf-91c961e6116e" containerID="e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc" exitCode=0 Nov 04 05:47:26.292715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.292561 5104 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" Nov 04 05:47:26.292715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.292590 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" event={"ID":"5255c316-71d7-4bca-9daf-91c961e6116e","Type":"ContainerDied","Data":"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc"} Nov 04 05:47:26.292715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.292642 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr" event={"ID":"5255c316-71d7-4bca-9daf-91c961e6116e","Type":"ContainerDied","Data":"01210751d92205591baf7c6a04330b86d847095a3761b0f92db6add5e118ffc2"} Nov 04 05:47:26.292715 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.292663 5104 scope.go:122] "RemoveContainer" containerID="e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc" Nov 04 05:47:26.301853 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.301835 5104 scope.go:122] "RemoveContainer" containerID="e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc" Nov 04 05:47:26.302113 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:26.302093 5104 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc\": container with ID starting with e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc not found: ID does not exist" containerID="e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc" Nov 04 05:47:26.302186 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.302123 5104 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc"} err="failed to get container status \"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc\": rpc error: code = NotFound desc = could not find container \"e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc\": container with ID starting with e3b79039c2d782561b2345767d5f0a280a8cb071fed6d24495c408db5279c5cc not found: ID does not exist" Nov 04 05:47:26.474806 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.474776 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:26.530335 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.530303 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-4fxvr"] Nov 04 05:47:26.547943 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.547911 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" path="/var/lib/kubelet/pods/5255c316-71d7-4bca-9daf-91c961e6116e/volumes" Nov 04 05:47:26.548368 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:26.548354 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" path="/var/lib/kubelet/pods/8109cf5d-32e0-42da-9369-ba00a8207a8c/volumes" Nov 04 05:47:30.490818 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:47:30.490781 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:47:34.368754 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:34.368715 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-tn2gh_ac436778-eb3c-42c3-8187-fe3f0289957a/manager/0.log" Nov 04 05:47:35.168042 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:35.168011 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-flnsj_6830b443-72e9-487f-8754-69dc783b70fd/manager/0.log" Nov 04 05:47:35.555212 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:35.555177 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:47:36.756676 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:36.756639 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:36.964712 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:36.964680 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:47:38.086948 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:38.086917 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-10107d08-q5zzt_1be6daf6-f6b1-40c8-8dfc-1ce90443af6f/revoke-keys/0.log" Nov 04 05:47:39.494339 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:39.494303 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7611e660-hqd2t_d829a495-9044-425f-af68-6b0f66ab99d1/revoke-keys/0.log" Nov 04 05:47:41.383404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:41.383366 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-bc2r4_e1efd7bb-616c-43c9-bd7c-aad1115a978c/postgres/0.log" Nov 04 05:47:43.446300 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:43.446268 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-667656475-7s2pb_f9fe744b-4bbe-4b56-a3a7-a10afc8688a9/authorino/0.log" Nov 04 05:47:43.839109 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:43.839077 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-mwkgq_7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6/manager/0.log" Nov 04 05:47:44.092884 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:44.092812 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-djtz5_a4582504-452c-4a5d-a936-7218745f84b1/manager/0.log" Nov 04 05:47:44.334825 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:44.334797 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vhlkv_f57d437b-159e-4b38-b09b-f3d092f3cf26/kuadrant-console-plugin/0.log" Nov 04 05:47:45.101850 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:45.101812 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-b7r22_67126a5b-679c-4a72-8803-d818835b6869/manager/0.log" Nov 04 05:47:45.581528 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:45.581494 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-mlw5m_48ae5a00-e13e-4f71-ab93-07b8d6fa5716/manager/1.log" Nov 04 05:47:46.330718 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:46.330686 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-mlw5m_48ae5a00-e13e-4f71-ab93-07b8d6fa5716/manager/0.log" Nov 04 05:47:46.942771 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:46.942743 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-58c98c4485xfwsh_0eef5bbb-45d9-4ccd-b519-c8a2009b1704/istio-proxy/0.log" Nov 04 05:47:48.343288 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:48.343255 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-hrddl_b0935188-2146-4564-9576-30cd88fae725/istio-proxy/0.log" Nov 04 05:47:48.834906 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:48.834862 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-tmnh8_fb571d04-7026-4946-83dc-b487e5231e5a/istio-proxy/1.log" Nov 04 05:47:49.074845 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:49.074781 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf_fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d/istio-proxy/0.log" Nov 04 05:47:50.535736 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:50.535706 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-tmnh8_fb571d04-7026-4946-83dc-b487e5231e5a/istio-proxy/0.log" Nov 04 05:47:54.055157 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:47:54.055122 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-667656475-7s2pb_f9fe744b-4bbe-4b56-a3a7-a10afc8688a9/authorino/0.log" Nov 04 05:48:00.490642 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:48:00.490601 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:48:03.872647 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:03.872614 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-tn2gh_ac436778-eb3c-42c3-8187-fe3f0289957a/manager/0.log" Nov 04 05:48:04.199476 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:04.199395 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:48:04.410558 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:04.410525 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:48:04.499770 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:04.499741 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-flnsj_6830b443-72e9-487f-8754-69dc783b70fd/manager/0.log" Nov 04 05:48:04.980607 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:04.980577 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:48:07.051342 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:07.051307 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-10107d08-q5zzt_1be6daf6-f6b1-40c8-8dfc-1ce90443af6f/revoke-keys/0.log" Nov 04 05:48:07.596248 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:07.596190 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:48:08.548653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:08.548623 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7611e660-hqd2t_d829a495-9044-425f-af68-6b0f66ab99d1/revoke-keys/0.log" Nov 04 05:48:10.552445 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:10.552411 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-bc2r4_e1efd7bb-616c-43c9-bd7c-aad1115a978c/postgres/0.log" Nov 04 05:48:11.838664 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:11.838637 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-667656475-7s2pb_f9fe744b-4bbe-4b56-a3a7-a10afc8688a9/authorino/0.log" Nov 04 05:48:12.279052 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:12.279026 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-mwkgq_7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6/manager/0.log" Nov 04 05:48:12.554723 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:12.554649 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-djtz5_a4582504-452c-4a5d-a936-7218745f84b1/manager/0.log" Nov 04 05:48:12.802404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:12.802373 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vhlkv_f57d437b-159e-4b38-b09b-f3d092f3cf26/kuadrant-console-plugin/0.log" Nov 04 05:48:13.105648 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:13.105622 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-b7r22_67126a5b-679c-4a72-8803-d818835b6869/manager/0.log" Nov 04 05:48:13.804017 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:13.803986 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-mlw5m_48ae5a00-e13e-4f71-ab93-07b8d6fa5716/manager/1.log" Nov 04 05:48:14.419468 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:14.419439 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-58c98c4485xfwsh_0eef5bbb-45d9-4ccd-b519-c8a2009b1704/istio-proxy/0.log" Nov 04 05:48:15.824316 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:15.824286 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-hrddl_b0935188-2146-4564-9576-30cd88fae725/istio-proxy/0.log" Nov 04 05:48:16.463900 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:16.463875 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-tmnh8_fb571d04-7026-4946-83dc-b487e5231e5a/istio-proxy/1.log" Nov 04 05:48:16.778718 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:16.778690 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-75b555675f-hh8sf_fb9c5a2c-5528-4b8e-81ac-938f4c4d0e9d/istio-proxy/0.log" Nov 04 05:48:30.489818 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:48:30.489778 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:48:32.932181 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:32.932147 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:48:33.140274 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:33.140240 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:48:43.752874 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:43.752840 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:48:44.348879 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:44.348850 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:48:57.523689 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:48:57.523611 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:00.489934 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:49:00.489899 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:49:07.972920 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:07.972887 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:17.728032 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:17.727997 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:17.841096 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:17.841063 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:49:18.406516 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:18.406483 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d77d66d96-nq9gq_c93362fa-d4a4-4786-b27a-c46db1b6084f/manager/0.log" Nov 04 05:49:19.975384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:19.975350 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:20.189384 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:20.189351 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:20.717753 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:20.717720 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:20.923522 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:20.923491 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:29.744810 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:29.744783 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dz6r4_54f8b1f3-a80d-47a0-bbb8-524762fdd08e/global-pull-secret-syncer/0.log" Nov 04 05:49:30.491433 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:49:30.491401 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:49:30.690335 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:30.690292 5104 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" limit=1024 Nov 04 05:49:31.049195 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:31.049162 5104 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt"] Nov 04 05:49:31.104628 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:31.104595 5104 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10107d08-q5zzt"] Nov 04 05:49:32.455610 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:32.455581 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qgzd8_535c0ff7-d319-49dd-a9f1-39b2179b9a1b/konnectivity-agent/0.log" Nov 04 05:49:32.548404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:32.548369 5104 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1be6daf6-f6b1-40c8-8dfc-1ce90443af6f" path="/var/lib/kubelet/pods/1be6daf6-f6b1-40c8-8dfc-1ce90443af6f/volumes" Nov 04 05:49:34.999864 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:34.999834 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-34.us-west-2.compute.internal_fd7b8286909c543935667f4273478b4f/haproxy/0.log" Nov 04 05:49:39.798302 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:39.798268 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-667656475-7s2pb_f9fe744b-4bbe-4b56-a3a7-a10afc8688a9/authorino/0.log" Nov 04 05:49:40.037248 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037207 5104 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28"] Nov 04 05:49:40.037593 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037579 5104 cpu_manager.go:414] "containerMap: removing container" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" Nov 04 05:49:40.037639 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037594 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" Nov 04 05:49:40.037639 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037610 5104 cpu_manager.go:414] "containerMap: removing container" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" Nov 04 05:49:40.037639 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037615 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" Nov 04 05:49:40.037639 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037631 5104 cpu_manager.go:414] "containerMap: removing container" podUID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" containerName="istio-proxy" Nov 04 05:49:40.037639 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037636 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037644 5104 cpu_manager.go:414] "containerMap: removing container" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037649 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037657 5104 cpu_manager.go:414] "containerMap: removing container" podUID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037662 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037671 5104 cpu_manager.go:414] "containerMap: removing container" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037678 5104 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037741 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="d66cbeee-0751-4aca-af9d-73e1c7c77901" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037752 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="5255c316-71d7-4bca-9daf-91c961e6116e" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037759 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="8109cf5d-32e0-42da-9369-ba00a8207a8c" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037766 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="70f4221b-5705-4352-8bbe-b5cf3c2dba4c" containerName="istio-proxy" Nov 04 05:49:40.037804 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.037771 5104 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f87b91a-8f51-48bc-ac01-e2c9b883cc14" containerName="istio-proxy" Nov 04 05:49:40.040866 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.040844 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.077857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.077785 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m45q2\" (UniqueName: \"kubernetes.io/projected/264479ae-cbbb-4f08-92c7-03d2bdaf504d-kube-api-access-m45q2\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.077857 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.077830 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-lib-modules\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.078014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.077920 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-sys\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.078014 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.077978 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-podres\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.078084 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.078043 5104 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-proc\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.100574 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.100544 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"kube-root-ca.crt\"" Nov 04 05:49:40.156092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.156055 5104 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"openshift-service-ca.crt\"" Nov 04 05:49:40.156092 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.156086 5104 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wtjmg\"/\"default-dockercfg-l9jq4\"" Nov 04 05:49:40.157974 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.157953 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28"] Nov 04 05:49:40.178880 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.178852 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-podres\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.178892 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-proc\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.178994 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m45q2\" (UniqueName: \"kubernetes.io/projected/264479ae-cbbb-4f08-92c7-03d2bdaf504d-kube-api-access-m45q2\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179007 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.178994 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-podres\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.179012 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-lib-modules\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.179038 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-proc\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.179083 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-lib-modules\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179113 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.179089 5104 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-sys\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.179263 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.179130 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/264479ae-cbbb-4f08-92c7-03d2bdaf504d-sys\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.245412 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.245389 5104 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m45q2\" (UniqueName: \"kubernetes.io/projected/264479ae-cbbb-4f08-92c7-03d2bdaf504d-kube-api-access-m45q2\") pod \"perf-node-gather-daemonset-5sl28\" (UID: \"264479ae-cbbb-4f08-92c7-03d2bdaf504d\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.353160 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.353098 5104 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:40.734594 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.734573 5104 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28"] Nov 04 05:49:40.737158 ip-10-0-139-34 kubenswrapper[5104]: W1104 05:49:40.737123 5104 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod264479ae_cbbb_4f08_92c7_03d2bdaf504d.slice/crio-6b4b904591d9672486e991d8a23fc8b2b38882c570aa567350805a1a17dd47ba WatchSource:0}: Error finding container 6b4b904591d9672486e991d8a23fc8b2b38882c570aa567350805a1a17dd47ba: Status 404 returned error can't find the container with id 6b4b904591d9672486e991d8a23fc8b2b38882c570aa567350805a1a17dd47ba Nov 04 05:49:40.783653 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.783631 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-mwkgq_7bcc7f7a-bdf3-48b8-9cbd-30da2838c3f6/manager/0.log" Nov 04 05:49:40.971760 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.971722 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" event={"ID":"264479ae-cbbb-4f08-92c7-03d2bdaf504d","Type":"ContainerStarted","Data":"4b930c1991116fd1cb37373c4f3823c16542b779bfd477c237a6aff151845b16"} Nov 04 05:49:40.972151 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.971783 5104 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" event={"ID":"264479ae-cbbb-4f08-92c7-03d2bdaf504d","Type":"ContainerStarted","Data":"6b4b904591d9672486e991d8a23fc8b2b38882c570aa567350805a1a17dd47ba"} Nov 04 05:49:40.972151 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:40.971815 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:41.098919 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:41.098810 5104 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" podStartSLOduration=2.098795455 podStartE2EDuration="2.098795455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:49:41.098437336 +0000 UTC m=+1318.662820635" watchObservedRunningTime="2026-11-04 05:49:41.098795455 +0000 UTC m=+1318.663178754" Nov 04 05:49:41.254093 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:41.254065 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-djtz5_a4582504-452c-4a5d-a936-7218745f84b1/manager/0.log" Nov 04 05:49:41.722166 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:41.722139 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-vhlkv_f57d437b-159e-4b38-b09b-f3d092f3cf26/kuadrant-console-plugin/0.log" Nov 04 05:49:43.225729 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:43.225699 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-b7r22_67126a5b-679c-4a72-8803-d818835b6869/manager/0.log" Nov 04 05:49:44.168137 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:44.168103 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-mlw5m_48ae5a00-e13e-4f71-ab93-07b8d6fa5716/manager/1.log" Nov 04 05:49:44.169404 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:44.169383 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-mlw5m_48ae5a00-e13e-4f71-ab93-07b8d6fa5716/manager/0.log" Nov 04 05:49:46.986735 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:46.986708 5104 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-5sl28" Nov 04 05:49:55.769727 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:55.769698 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzr9z_64dba94c-196c-42cc-953a-5fc0864b7614/node-exporter/0.log" Nov 04 05:49:56.235745 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:56.235709 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzr9z_64dba94c-196c-42cc-953a-5fc0864b7614/kube-rbac-proxy/0.log" Nov 04 05:49:56.701001 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:49:56.700926 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzr9z_64dba94c-196c-42cc-953a-5fc0864b7614/init-textfile/0.log" Nov 04 05:50:00.490030 ip-10-0-139-34 kubenswrapper[5104]: E1104 05:50:00.489987 5104 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8hnv7" containerName="extractor" Nov 04 05:50:01.855564 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:50:01.855523 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-cw92n_70b57d48-f09f-4b30-ac48-63e558200561/prometheus-operator/0.log" Nov 04 05:50:02.316936 ip-10-0-139-34 kubenswrapper[5104]: I1104 05:50:02.316904 5104 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-cw92n_70b57d48-f09f-4b30-ac48-63e558200561/kube-rbac-proxy/0.log"